Automatic data modeling and optimizing system and method fusing knowledge graph and ChatBI

Abstract

The invention discloses an automatic data modeling and optimizing system and method fusing a knowledge graph and ChatBI, and relates to the technical field of data analysis. In order to solve the problems of high interaction threshold and insufficient analysis depth of a traditional BI tool, the scheme adopted by the invention comprises a data layer which has the capabilities of multi-source data access, domain knowledge graph construction, intelligent mapping recommendation, federal calculation and data mild governance, and realizes data integration and semantic unification; the analysis layer realizes accurate conversion from a natural language to an SQL and semantic reasoning of a knowledge graph through graph vectorization, multi-model cooperation, natural language understanding, intelligent SQL generation and graph dynamic updating, and supports efficient data query and analysis; and the application layer has the functions of natural language interaction, visual recommendation and generation, root cause analysis and intelligent early warning, and provides a visual interaction interface and data display service for a user. According to the invention, intelligent data analysis and visualization based on natural language interaction can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular to an automatic data modeling and optimization system and method fusing knowledge graph and ChatBI. BACKGROUND

[0002] The traditional business intelligence (BI) tools have the following prominent pain points:

[0003] Insufficient data dimension and display flexibility: the traditional BI application presentation mode is standardized and solidified, and the display content cannot be changed in time with the adjustment of production business. Ultimately, these application displays cannot meet the business changes and may be abandoned or redeveloped.

[0004] Weak data problem visualization and decision support: the traditional BI application usually needs to be customized and developed with great cost to develop data alarm rules and data applications to highlight the problems in the data. Due to the difficulty in enumerating all scenarios, the data application page is mediocre, it is difficult to quickly locate data problems, and it cannot support accurate decision-making.

[0005] High development cost and low efficiency: the traditional BI tool usually develops in a template mode, which needs to adjust the data content and style layout. The actual human and time costs have not been significantly reduced. After the business personnel propose the demand, the IT personnel usually need several days or even several weeks of development period.

[0006] Limited analysis depth: the traditional BI tool can only meet the correlation analysis, lacks the ability to mine causal relationships, and is difficult to support complex root cause analysis and predictive analysis.

[0007] Low user participation: the traditional data modeling highly depends on professional personnel, and the business personnel are difficult to directly participate, resulting in deviation between the model and the actual demand, and the analysis result has high understanding threshold for non-technical personnel.

[0008] In view of the above-mentioned pain points, an automatic data modeling and optimization system and method fusing knowledge graph and ChatBI are proposed, which improves the accuracy, efficiency and analysis depth of ChatBI by means of the semantic representation and reasoning ability of knowledge graph, and realizes real intelligent data analysis. SUMMARY

[0009] The present application provides an automatic data modeling and optimization system and method fusing knowledge graph and ChatBI to solve the problems of single data dimension, low development efficiency, high interaction threshold and insufficient analysis depth of the traditional BI tool, and realizes intelligent data analysis and visualization based on natural language interaction.

[0010] In a first aspect, the application provides an automatic data modeling and optimization system integrating a knowledge graph and a ChatBI, which solves the above technical problems by adopting the following technical solutions:

[0011] An automatic data modeling and optimization system integrating a knowledge graph and a ChatBI, which comprises:

[0012] A data layer, which has the capabilities of multi-source data access, domain knowledge graph construction, intelligent mapping recommendation, federated computing, and data light governance, and realizes data integration and semantic unification;

[0013] An analysis layer, which realizes accurate conversion from natural language to SQL and semantic reasoning of the knowledge graph through graph vectorization, multi-model collaboration, natural language understanding, intelligent SQL generation, and dynamic graph updating, and supports efficient data query and analysis;

[0014] An application layer, which has the functions of natural language interaction, visualization recommendation and generation, root cause analysis, and intelligent early warning, and provides users with an intuitive and convenient interactive interface and data display service, and improves the ease of use and depth of data analysis.

[0015] Optionally, the data layer comprises:

[0016] A data access and integration module, which is used to access multi-source heterogeneous data through an ETL tool, preliminarily clean the accessed data, and realize centralized integration of structured and unstructured data;

[0017] A graph construction module, which is used to construct a static knowledge graph recording fixed entity relationships and a dynamic knowledge graph recording changes of entity relationships over time based on a domain ontology model containing categories, relationships, attributes, and constraint rules;

[0018] An intelligent mapping recommendation module, which is used to analyze the matching relationship between data patterns and the domain ontology model through entity vector similarity, attribute matching, and pattern matching algorithms, and automatically generate mapping recommendations to optimize the entity relationship definitions of the static knowledge graph and the dynamic knowledge graph;

[0019] A federated computing engine module, which is used to realize efficient integration and computing across data sources through distributed data query based on the optimized static knowledge graph and dynamic knowledge graph;

[0020] A data light governance module, which is used to provide data quality evaluation, data bloodline tracking, and data security control functions, and guarantee the accuracy and security of data.

[0021] Further optionally, the analysis layer comprises:

[0022] The graph vectorization module is configured to convert entities and relationships in the static knowledge graph and the dynamic knowledge graph into vector representations by using a graph embedding technology and a text embedding technology, wherein the entity vector fuses structural features and descriptive text information, and the relationship vector reflects semantic association.

[0023] The multi-model hybrid module is configured to provide model capability support for natural language processing and SQL generation by using a division mode of a large language model and a lightweight special-purpose model, wherein the large language model is responsible for natural language understanding and SQL framework generation, and the lightweight special-purpose model is responsible for processing data prediction and execution path optimization.

[0024] The natural language understanding module is configured to integrate a vector database to construct a retrieval enhancement generation system by relying on vector data generated by the graph vectorization module and the large language model in the multi-model hybrid module; and to implement efficient retrieval by storing knowledge graph vectors and historical SQL statement vectors in the vector database, integrate retrieval information as enhanced context, improve the semantic understanding accuracy of the large language model for user problems, and lay a foundation for subsequent SQL generation.

[0025] The intelligent SQL generation module is configured to automatically generate SQL query statements conforming to business logic by using a pre-trained natural language to SQL conversion model based on entity relationship vector representations of the graph vectorization module, a division mode of the multi-model hybrid module, and user intent analysis results output by the natural language understanding module, and under the constraint of the static knowledge graph and the dynamic knowledge graph.

[0026] The graph dynamic updating module is configured to define a trigger mechanism based on user query frequency, data change events, or business indicator abnormalities, optimize vector representation of incremental data in the graph vectorization module by using a time sequence perception model, ensure real-time synchronization of the dynamic knowledge graph with business changes, and feed back the optimization results to the retrieval enhancement generation system of the natural language understanding module and the statement generation logic of the intelligent SQL generation module to form a dynamic closed loop.

[0027] Preferably, the large language model is a GPT model or an LLaMA model that has natural language deep understanding and structured content generation capabilities, and is configured to receive a natural language instruction input by a user, analyze semantic information and task targets in the instruction, and generate structured content conforming to grammar rules and logical structures based on pre-trained domain knowledge.

[0028] The lightweight special model involved is a small model designed for data prediction or path optimization tasks, which is configured to: receive structured content output by the large language model as input, calculate feature parameters of the input based on domain-specific training data and output prediction results in a data prediction scenario; in a path optimization scenario, algorithm iteration optimization is performed on the input preliminary execution path to simplify the process; through cooperation with the large language model, a technical link from natural language instruction analysis to efficient execution of special tasks is formed, improving the accuracy and efficiency of task processing.

[0029] Further optionally, the application layer involved specifically includes:

[0030] A natural language interaction module is configured to provide a natural language interaction interface, support users to ask questions about report data in the form of voice or text, and realize real-time interaction of data;

[0031] A visual recommendation module is configured to automatically match and recommend an appropriate visualization chart based on the semantic content of the user query, the entity relationship type of the static knowledge graph and the dynamic knowledge graph, and the pre-established entity relationship type and chart type mapping table;

[0032] An intelligent chart generation module is configured to convert the SQL query result into a diversified visualization chart according to the recommendation result, and optimize the display mode in combination with the semantic information of the static knowledge graph and the dynamic knowledge graph to highlight key data problems;

[0033] An analysis and early warning module is configured to rely on the semantic reasoning capability of the static knowledge graph and the dynamic knowledge graph to carry out root cause analysis and realize intelligent early warning of data anomalies through path search and causal relationship inference based on the generated chart and associated data, thereby assisting users to quickly locate the root cause of the problem to avoid potential risks in advance.

[0034] In a second aspect, an automatic data modeling and optimization method fusing knowledge graphs and ChatBI, which solves the above technical problems by adopting the following technical solutions:

[0035] An automatic data modeling and optimization method fusing knowledge graphs and ChatBI, which involves a data layer, an analysis layer and an application layer, and specifically includes the following steps:

[0036] S1, the data layer first accesses multi-source data, synchronously constructs a domain knowledge graph, realizes data correlation matching through intelligent mapping recommendation, guarantees cross-source data processing security in combination with federated computing, simultaneously carries out data mild governance, forms an integrated and semantically unified basic data system, and provides standardized data support for the subsequent analysis layer;

[0037] S2, based on the knowledge graph formed in step S1 and the integrated data, the analysis layer first performs graph vectorization processing, and then enhances the natural language understanding capability through multi-model collaboration to generate intelligent SQL to realize accurate conversion of natural language to data query instructions, synchronously updates the knowledge graph to optimize the semantic reasoning effect, and provides core technical support for efficient data query and analysis;

[0038] S3, relying on the query results and reasoning conclusions output by the analysis layer, the application layer provides a natural language interaction interface, automatically recommends and generates visual content, carries out root cause analysis and intelligent early warning in combination with the knowledge graph, and finally provides users with intuitive and convenient interactive experience and deep data analysis services, and improves the ease of use and decision-making value of data analysis.

[0039] Optionally, the data layer involved specifically performs the following operations:

[0040] S1.1, access multi-source heterogeneous data through an ETL tool, preliminarily clean the accessed data, and realize centralized integration of structured and unstructured data;

[0041] S1.2, based on a domain ontology model containing categories, relationships, attributes and constraint rules, a static knowledge graph recording fixed entity relationships and a dynamic knowledge graph recording changes of entity relationships over time are constructed;

[0042] S1.3, through entity vector similarity, attribute matching and pattern matching algorithms, the matching relationship between the data mode and the domain ontology model is analyzed, and mapping recommendation is automatically generated to optimize the entity relationship of the static knowledge graph and the dynamic knowledge graph;

[0043] S1.4, based on the optimized static knowledge graph and dynamic knowledge graph, distributed data query is used to realize efficient integration and calculation across data sources;

[0044] S1.5, through data quality evaluation, data bloodline tracking and data security control, the accuracy and safety of the data in each link are ensured in real time, and reliable support is provided for the entire data processing link.

[0045] Further optionally, the analysis layer involved specifically performs the following operations:

[0046] S2.1, graph embedding technology and text embedding technology are used to convert entities and relationships in the static knowledge graph and the dynamic knowledge graph into vector representations, wherein the entity vector fuses structure features and description text information, and the relationship vector reflects semantic association;

[0047] S2.2, a division mode of large language model and light-weighted special model is adopted to provide model capability support for natural language processing and SQL generation, wherein the large language model is responsible for natural language understanding and SQL framework generation, and the light-weighted special model is responsible for processing data prediction and execution path optimization;

[0048] S2.3, based on the vector representation generated in step S2.1 and the large language model in step S2.2, an integrated vector database construction retrieval enhancement generation system is constructed, the entity and relationship vectors of the knowledge graph and the historical SQL statement vectors are stored in the vector database to realize efficient retrieval, the retrieval information is integrated as enhanced context to improve the semantic understanding accuracy of the large language model for user problems, and a foundation is laid for subsequent SQL generation;

[0049] S2.4, based on the vector representation generated in step S2.1, the division mode of step S2.2, and the user intent analysis result output by step S2.3, with the constraint action of static knowledge graph and dynamic knowledge graph, a pre-trained natural language to SQL conversion model is used to automatically generate SQL query statements conforming to business logic;

[0050] S2.5, a trigger mechanism based on user query frequency, data change event or business index exception is defined, and a time sequence perception model is combined to optimize the vector representation of the incremental data in step S2.1, to ensure that the dynamic knowledge graph is synchronized with the business changes in real time, and the optimization result is fed back to the retrieval enhancement generation system in step S2.3 and the SQL query statement generation link in step S2.4, forming a dynamic closed loop.

[0051] Preferably, the large language model involved is a GPT model or LLaMA model with natural language deep understanding and structured content generation capability, which is configured to: receive a user input natural language instruction, parse the semantic information and task target in the instruction, and generate structured content conforming to grammar rules and logical structure based on pre-trained domain knowledge;

[0052] The light-weighted special model involved is a small model designed for data prediction or path optimization tasks, which is configured to: receive the structured content output by the large language model as input, in the data prediction scenario, calculate the feature parameters of the input based on domain-specific training data and output the prediction result; in the path optimization scenario, algorithm iteration optimization is performed on the preliminary execution path of the input to simplify the process; through cooperation with the large language model, a technical link from natural language instruction analysis to efficient execution of special tasks is formed, and the accuracy and efficiency of task processing are improved.

[0053] Optionally, the application layer specifically performs the following operations:

[0054] S3.1, a natural language interaction interface is provided, supporting users to ask report data in the form of voice or text, realizing real-time interaction of data;

[0055] S3.2, based on the semantic content of user query, combining the entity relationship type of knowledge graph, referring to the pre-established entity relationship type and chart type mapping table, automatically matching and recommending the adaptive visualization chart;

[0056] S3.3, according to the recommendation result, the SQL query result is directly converted into diversified visualization chart, and the semantic information of static knowledge graph and dynamic knowledge graph is combined to optimize the display mode, and the key data problem is highlighted;

[0057] S3.4, based on the generated chart and associated data, relying on the semantic reasoning ability of static knowledge graph and dynamic knowledge graph, through path search and causal relationship inference, the root cause analysis of data anomaly is carried out and intelligent early warning is realized, which assists users to quickly locate the problem source to avoid potential risks in advance.

[0058] The automatic data modeling and optimization system and method fusing knowledge graph and ChatBI of the application have the beneficial effects compared with the prior art:

[0059] 1, the three-layer architecture of data layer, analysis layer and application layer is constructed, the intelligent data analysis and visualization based on natural language interaction are realized, the semantic representation ability of knowledge graph is used, and the problems of single data dimension, low development efficiency, high interaction threshold and insufficient analysis depth of traditional BI tools are solved;

[0060] 2, the data layer of the application realizes semantic unification and dynamic integration of multi-source heterogeneous data through the intelligent mapping recommendation mechanism of knowledge graph; the analysis layer combines RAG technology and entity relationship vector representation of knowledge graph to optimize the conversion accuracy from natural language to SQL; the application layer realizes intelligent chart generation and root cause analysis through the visualization recommendation logic driven by knowledge graph; the use threshold of data analysis can be significantly reduced, and the data query efficiency and analysis depth can be improved;

[0061] 3, the application is suitable for data analysis scenes of financial, medical, manufacturing, retail and other industries, can realize complex data query, analysis and visualization through natural language interaction, reduce the data analysis threshold, and improve the analysis efficiency and accuracy;

[0062] 4、The application can reduce the data analysis threshold, so that non-technical business personnel can easily obtain data insights; can improve data analysis efficiency, realize fast response from problem to answer; can deepen data analysis capability, support causal reasoning and root cause analysis, provide deeper data insights; realize automatic construction and optimization of data model, improve the flexibility and adaptability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0063] FIG. 1 is an implementation architecture diagram of the first embodiment of the application; Figure 1

[0064] FIG. 2 is a method flowchart of the second embodiment of the application. Figure 2 DETAILED DESCRIPTION

[0065] In order to make the technical solutions, technical problems solved and technical effects of the application clearer and more apparent, the technical solutions of the application will be described in detail below in combination with specific embodiments.

[0066] Embodiment 1

[0067] In combination with FIG. 1, the embodiment proposes an automatic data modeling and optimization system fusing knowledge graph and ChatBI, which comprises: Figure 1 A data layer, which has the capabilities of multi-source data access, domain knowledge graph construction, intelligent mapping recommendation, federated computing and data light governance, realizes data integration and semantic unification;

[0068] An analysis layer, which realizes accurate conversion from natural language to SQL and semantic reasoning of knowledge graph through graph vectorization, multi-model collaboration, natural language understanding, intelligent SQL generation and dynamic graph updating, supports efficient data query and analysis.

[0069] An application layer, which has the functions of natural language interaction, visualization recommendation and generation, root cause analysis and intelligent early warning, provides users with intuitive and convenient interactive interface and data display services, and improves the ease of use and depth of data analysis.

[0070] In the embodiment, the data layer specifically comprises:

[0071] A data access and integration module, which is used for accessing relational databases, NoSQL databases, file systems, APIs and other multi-source heterogeneous data through ETL tools, preliminarily cleaning the accessed data, and realizing centralized integration of structured and unstructured data;

[0072]

[0073] ​​​A graph construction module is configured to construct a static knowledge graph recording fixed entity relationships and a dynamic knowledge graph recording changes in entity relationships over time based on a domain ontology model containing categories, relationships, attributes, and constraint rules.

[0074] An intelligent mapping recommendation module is configured to analyze the matching relationship between data patterns and the domain ontology model through entity vector similarity (such as the TransE algorithm), attribute matching, and pattern matching algorithms (such as the correspondence between tables and ontology concepts and fields and attributes), and automatically generate mapping recommendations to optimize the definition of entity relationships in the static knowledge graph and the dynamic knowledge graph.

[0075] A federated computing engine module is configured to implement efficient integration and computation across data sources through distributed data queries based on the optimized static knowledge graph and dynamic knowledge graph.

[0076] A data light governance module is configured to provide data quality assessment, data lineage tracking, and data security control functions to ensure the accuracy and security of data.

[0077] In this embodiment, the parsing layer specifically includes:

[0078] A graph vectorization module is configured to use graph embedding techniques (such as TransE, GraphSAGE, etc.) and text embedding techniques (such as BERT) to convert entities and relationships in the static knowledge graph and the dynamic knowledge graph into vector representations, where entity vectors integrate structural features and descriptive text information, and relationship vectors reflect semantic associations.

[0079] A multi-model hybrid module is configured to use a division-of-labor mode of large language models and lightweight specialized models to provide model capability support for natural language processing and SQL generation, where the large language models are responsible for natural language understanding and SQL framework generation, and the lightweight specialized models are responsible for data prediction and execution path optimization.

[0080] A natural language understanding module is configured to rely on vector data generated by the graph vectorization module and large language models (such as GPT, Claude) in the multi-model hybrid module, integrate vector databases (such as Milvus, ChromaDB) to build a retrieval augmented generation (RAG) system; and is configured to implement efficient retrieval by storing knowledge graph vectors and historical SQL statement vectors in the vector database, integrate retrieval information into augmented context, improve the semantic understanding accuracy of the large language model for user questions, and lay the foundation for subsequent SQL generation.

[0081] An intelligent SQL generation module is configured to generate SQL query statements in accordance with business logic based on the entity relationship vector representation of the graph vectorization module, the division mode of the multi-model hybrid module, and the user intent analysis result output by the natural language understanding module, with the aid of the constraint effect of the static knowledge graph and the dynamic knowledge graph, and by using a pre-trained natural language to SQL conversion model;

[0082] A graph dynamic updating module is configured to define a trigger mechanism based on user query frequency, data change events (such as adding a field), or business index abnormalities, and to optimize the vector representation of incremental data in the graph vectorization module in combination with a time sequence perception model (TAE), so as to ensure that the dynamic knowledge graph is synchronized with business changes in real time, and the optimization result is fed back to the retrieval enhancement generation (RAG) system of the natural language understanding module and the statement generation logic of the intelligent SQL generation module, forming a dynamic closed loop.

[0083] It should be noted that the large language model involved is a GPT model or LLaMA model that has the ability of natural language deep understanding and structured content generation, which is configured to receive a natural language instruction input by a user, analyze semantic information and task targets in the instruction, and generate structured content in accordance with grammar rules and logical structures based on domain knowledge obtained through pre-training;

[0084] The lightweight special-purpose model involved is a small model designed for data prediction or path optimization tasks, which is configured to receive the structured content output by the large language model as input, calculate feature parameters of the input based on domain-specific training data and output prediction results in a data prediction scenario, and iteratively optimize the preliminary execution path of the input to simplify the process in a path optimization scenario. Through cooperation with the large language model, a technical link from natural language instruction analysis to efficient execution of special tasks is formed, improving the accuracy and efficiency of task processing.

[0085] In this embodiment, the application layer specifically includes:

[0086] A natural language interaction module is configured to provide a natural language interaction interface, support the user to ask questions about report data in the form of voice or text, and realize real-time interaction of data;

[0087] A visual recommendation module is configured to automatically match and recommend an appropriate visual chart based on the semantic content of the user query, in combination with the entity relationship types of the static knowledge graph and the dynamic knowledge graph, and with reference to a pre-established mapping table of entity relationship types and chart types;

[0088] The intelligent chart generation module is used to directly convert SQL query results into diverse visualization charts (including bar charts, pie charts, funnel charts, GIS maps, etc.) based on recommendations. It also optimizes the display method by combining the semantic information of static and dynamic knowledge graphs to highlight key data issues.

[0089] The analysis and early warning module is used to perform root cause analysis on data anomalies based on generated charts and related data, relying on the semantic reasoning capabilities of static and dynamic knowledge graphs, and through path search and causal relationship inference, and to achieve intelligent early warning, so as to help users quickly locate the root cause of the problem and avoid potential risks in advance.

[0090] Example 2:

[0091] Combined with appendix Figure 2 This embodiment proposes an automated data modeling and optimization method that integrates knowledge graphs and ChatBI. Its implementation involves a data layer, a parsing layer, and an application layer, and specifically includes the following steps:

[0092] S1, the data layer first accesses multi-source data, synchronously constructs a domain knowledge graph, realizes data association and matching through intelligent mapping and recommendation, combines federated computing to ensure the security of cross-source data processing, and carries out light data governance to form an integrated and semantically unified basic data system, providing standardized data support for the subsequent parsing layer.

[0093] The data layer involved performs the following operations:

[0094] S1.1. Use ETL tools to access heterogeneous data from multiple sources such as relational databases, NoSQL databases, file systems, and APIs, perform preliminary cleaning on the accessed data, and achieve centralized integration of structured and unstructured data;

[0095] S1.2 Based on a domain ontology model that includes categories, relationships, attributes, and constraint rules, construct a static knowledge graph that records fixed entity relationships and a dynamic knowledge graph that records changes in entity relationships over time.

[0096] S1.3. By using entity vector similarity (such as the TransE algorithm), attribute matching and pattern matching algorithms (such as the correspondence between tables and ontology concepts, fields and attributes), analyze the matching relationship between data patterns and domain ontology models, and automatically generate mapping recommendations to optimize entity relationships in static knowledge graphs and dynamic knowledge graphs.

[0097] S1.4 Based on the optimized static and dynamic knowledge graphs, efficient integration and computation across data sources are achieved through distributed data querying;

[0098] S1.5, by data quality assessment, data blood origin tracking and data security control, real-time guarantee the accuracy and safety of data in each link, and provide reliable support for the whole data processing link.

[0099] S2, based on the knowledge graph formed in step S1 and the integrated data, the analysis layer first performs graph vectorization processing, then strengthens the natural language understanding ability through multi-model cooperation, generates intelligent SQL to realize accurate conversion of natural language to data query instruction, synchronously updates the knowledge graph to optimize the semantic reasoning effect, and provides core technical support for efficient data query and analysis.

[0100] The analysis layer involved specifically performs the following operations:

[0101] S2.1, adopt graph embedding technology (such as TransE, GraphSAGE, etc.) and text embedding technology (such as BERT), convert entities and relationships in static knowledge graph and dynamic knowledge graph into vector representation, wherein the entity vector fuses structure features and description text information, and the relationship vector reflects semantic association;

[0102] S2.2, adopt the division mode of large language model and lightweight special-purpose model to provide model capability support for natural language processing and SQL generation, wherein the large language model is responsible for natural language understanding and SQL framework generation, and the lightweight special-purpose model is responsible for processing data prediction and execution path optimization;

[0103] S2.3, based on the vector representation generated in step S2.1 and the large language model (such as GPT, Claude) in step S2.2, integrate vector database (such as Milvus, ChromaDB) to build a retrieval augmented generation (RAG) system, store entity and relationship vectors of knowledge graph and historical SQL statement vectors in vector database to realize efficient retrieval, integrate retrieval information as augmented context, improve the semantic understanding accuracy of large language model to user's question, and lay a foundation for subsequent SQL generation;

[0104] S2.4, based on the vector representation generated in step S2.1, the division mode of step S2.2 (especially the path optimization capability of lightweight special-purpose model), and the user intention analysis result output in step S2.3, with the constraint effect of static knowledge graph and dynamic knowledge graph, using the pre-trained natural language to SQL conversion model, automatically generate SQL query statement conforming to business logic;

[0105] S2.5, define a trigger mechanism based on user query frequency, data change events (such as adding fields) or business indicator abnormalities, combine the time sequence perception model (TAE) to optimize the vector representation of the incremental data in step S2.1, ensure that the dynamic knowledge graph is synchronized with the business changes in real time, and the optimization results are fed back to the retrieval enhancement generation (RAG) system in step S2.3 and the SQL query statement generation link in step S2.4, forming a dynamic closed loop.

[0106] It should be noted that the large language model involved is a GPT model or LLaMA model with natural language deep understanding and structured content generation capability, which is configured to: receive user input natural language instructions, parse the semantic information and task target in the instructions, and generate structured content conforming to grammar rules and logical structure based on pre-trained domain knowledge.

[0107] The lightweight special-purpose model involved is a small model designed for data prediction or path optimization tasks, which is configured to: receive the structured content output by the large language model as input, in the data prediction scenario, calculate the feature parameters of the input based on the domain-specific training data and output the prediction result; in the path optimization scenario, algorithm iteratively optimizes the preliminary execution path of the input to simplify the process; through cooperation with the large language model, a technical link from natural language instruction analysis to efficient execution of special tasks is formed, improving the accuracy and efficiency of task processing.

[0108] S3, relying on the query results and reasoning conclusions output by the analysis layer, the application layer provides a natural language interaction interface, automatically recommends and generates visual content, conducts root cause analysis and intelligent early warning based on the knowledge graph, and finally provides users with intuitive and convenient interactive experience and deep data analysis services, improving the ease of use and decision-making value of data analysis.

[0109] The application layer involved specifically performs the following operations:

[0110] S3.1, provide a natural language interaction interface, support users to ask questions about report data in the form of voice or text, realize real-time interaction of data;

[0111] S3.2, based on the semantic content of user queries, combined with the entity relationship types of the knowledge graph, referring to the pre-established entity relationship type and chart type mapping table, automatically matching and recommending suitable visual charts;

[0112] S3.3, according to the recommendation result, directly convert the SQL query result into diversified visual charts (including column chart, pie chart, funnel chart, GIS map, etc.), and combine the semantic information of static knowledge graph and dynamic knowledge graph to optimize the display mode, highlight the key data problems;

[0113] S3.4, based on the generated chart and associated data, relying on the semantic reasoning ability of the static knowledge graph and the dynamic knowledge graph, through path search and causal relationship inference, the root cause analysis of data anomaly is carried out and intelligent early warning is realized, which assists users to quickly locate the problem source to avoid potential risks in advance.

[0114] In conclusion, by using the automatic data modeling and optimization system and method fusing knowledge graph and ChatBI, through the construction of three-layer architecture of data layer, analysis layer and application layer, intelligent data analysis and visualization based on natural language interaction are realized, the semantic representation ability of knowledge graph is used, and the problems of single data dimension, low development efficiency, high interaction threshold and insufficient analysis depth of traditional BI tools are solved.

[0115] The above application specific examples have described the principles and implementation modes of the present application in detail, and these examples are only used to help understand the core technical content of the present application. Based on the above specific embodiments of the present application, any improvement and modification of the present application made by the person skilled in the art without departing from the principles of the present application shall fall within the patent protection scope of the present application.

Claims

1. An automated data modeling and optimization system integrating knowledge graphs and ChatBI, characterized in that, It includes: The data layer has the capabilities of multi-source data access, domain knowledge graph construction, intelligent mapping and recommendation, federated computing and light data governance, to achieve data integration and semantic unification; The parsing layer, through graph vectorization, multi-model collaboration, natural language understanding, intelligent SQL generation, and dynamic graph updates, achieves accurate conversion of natural language to SQL and semantic reasoning of knowledge graphs, supporting efficient data query and analysis. At the application layer, it features natural language interaction, visual recommendation and generation, root cause analysis, and intelligent early warning functions, providing users with an intuitive and convenient interactive interface and data display services, thereby improving the ease of use and depth of data analysis.

2. The automated data modeling and optimization system integrating knowledge graphs and ChatBI as described in claim 1, characterized in that, The data layer specifically includes: The data access and integration module is used to access multi-source heterogeneous data through ETL tools, perform preliminary cleaning on the accessed data, and achieve centralized integration of structured and unstructured data. The knowledge graph construction module is used to build static knowledge graphs that record fixed entity relationships and dynamic knowledge graphs that record changes in entity relationships over time, based on a domain ontology model that includes categories, relationships, attributes, and constraint rules. The intelligent mapping recommendation module is used to analyze the matching relationship between data patterns and domain ontology models through entity vector similarity, attribute matching and pattern matching algorithms, and automatically generate mapping recommendations to optimize the entity relationship definition of static knowledge graphs and dynamic knowledge graphs. The federated computing engine module is used to achieve efficient integration and computation across data sources through distributed data queries based on optimized static and dynamic knowledge graphs. The data light governance module provides data quality assessment, data lineage tracing, and data security control functions to ensure data accuracy and security.

3. The automated data modeling and optimization system integrating knowledge graphs and ChatBI as described in claim 2, characterized in that, The parsing layer specifically includes: The graph vectorization module is used to transform entities and relationships in static and dynamic knowledge graphs into vector representations using graph embedding and text embedding techniques. Entity vectors integrate structural features and descriptive text information, while relationship vectors reflect semantic associations. The multi-model hybrid module is used to adopt a division of labor mode of large language models and lightweight special-purpose models to provide model capability support for natural language processing and SQL generation. The large language model is responsible for natural language understanding and SQL framework generation, while the lightweight special-purpose model is responsible for data prediction and execution path optimization. The Natural Language Understanding module is used to integrate vector data generated by the knowledge graph vectorization module and the large language model in the multi-model hybrid module to build a retrieval enhancement generation system by integrating a vector database. It is used to achieve efficient retrieval by storing knowledge graph vectors and historical SQL statement vectors in the vector database, integrate retrieval information to enhance context, improve the semantic understanding accuracy of the large language model for user questions, and lay the understanding foundation for subsequent SQL generation. The intelligent SQL generation module is used to automatically generate SQL query statements that conform to business logic by leveraging the constraints of static and dynamic knowledge graphs and a pre-trained natural language to SQL conversion model, based on the entity relationship vector representation of the graph vectorization module, the division of labor mode of the multi-model hybrid module, and the user intent parsing results output by the natural language understanding module. The graph dynamic update module defines triggering mechanisms based on user query frequency, data change events, or abnormal business metrics. Combined with a time-series-aware model, it optimizes the vector representation of incremental data in the graph vectorization module, ensuring that the dynamic knowledge graph is synchronized with business changes in real time. The optimization results are also synchronously fed back to the retrieval enhancement generation system of the natural language understanding module and the statement generation logic of the intelligent SQL generation module, forming a dynamic closed loop.

4. The automated data modeling and optimization system integrating knowledge graphs and ChatBI as described in claim 3, characterized in that, The large language model is a GPT model or an LLaMA model with the ability to deeply understand natural language and generate structured content. It is configured to: receive natural language instructions input by the user, parse the semantic information and task objectives in the instructions, and generate structured content that conforms to grammatical rules and logical structure based on the domain knowledge obtained through pre-training. The lightweight dedicated model is a miniaturized model designed for data prediction or path optimization tasks. It is configured to: receive the structured content output by the large language model as input; in the data prediction scenario, calculate the feature parameters of the input based on domain-specific training data and output the prediction result; in the path optimization scenario, perform algorithmic iterative optimization on the initial execution path of the input to simplify the process; and through collaboration with the large language model, form a technical link from natural language instruction parsing to efficient execution of the dedicated task, thereby improving the accuracy and efficiency of task processing.

5. The automated data modeling and optimization system integrating knowledge graphs and ChatBI as described in claim 3, characterized in that, The application layer specifically includes: The natural language interaction module provides a natural language interaction interface, allowing users to ask questions about report data in the form of voice or text, and realize real-time data interaction. The visualization recommendation module is used to automatically match and recommend suitable visualization charts based on the semantic content of user queries, combined with the entity relationship types of static and dynamic knowledge graphs, and referring to a pre-established mapping table between entity relationship types and chart types. The intelligent chart generation module is used to directly convert SQL query results into diverse visualization charts based on recommendation results, and optimize the display method by combining the semantic information of static and dynamic knowledge graphs to highlight key data issues; The analysis and early warning module is used to perform root cause analysis on data anomalies based on generated charts and related data, relying on the semantic reasoning capabilities of static and dynamic knowledge graphs, and through path search and causal relationship inference, and to achieve intelligent early warning, so as to help users quickly locate the root cause of the problem and avoid potential risks in advance.

6. An automated data modeling and optimization method integrating knowledge graphs and ChatBI, characterized in that, Its implementation involves a data layer, a parsing layer, and an application layer, and specifically includes the following steps: S1. The data layer first accesses multi-source data, synchronously constructs a domain knowledge graph, realizes data association and matching through intelligent mapping and recommendation, combines federated computing to ensure the security of cross-source data processing, and carries out light data governance to form an integrated and semantically unified basic data system, providing standardized data support for the subsequent parsing layer. S2. Based on the knowledge graph and integrated data formed in step S1, the parsing layer first performs graph vectorization processing, then enhances natural language understanding capabilities through multi-model collaboration, generates intelligent SQL to achieve accurate conversion of natural language into data query commands, and synchronously updates the knowledge graph to optimize semantic reasoning effects, providing core technical support for efficient data query and analysis. S3. Based on the query results and reasoning conclusions output by the parsing layer, the application layer provides a natural language interaction interface, automatically recommends and generates visual content, and combines knowledge graphs to carry out root cause analysis and intelligent early warning. Ultimately, it provides users with an intuitive and convenient interactive experience and in-depth data analysis services, improving the usability and decision-making value of data analysis.

7. The automated data modeling and optimization system integrating knowledge graphs and ChatBI as described in claim 6, characterized in that, The data layer specifically performs the following operations: S1.

1. Access multi-source heterogeneous data through ETL tools, perform preliminary cleaning on the accessed data, and achieve centralized integration of structured and unstructured data; S1.2 Based on a domain ontology model that includes categories, relationships, attributes, and constraint rules, construct a static knowledge graph that records fixed entity relationships and a dynamic knowledge graph that records changes in entity relationships over time. S1.

3. By using entity vector similarity, attribute matching, and pattern matching algorithms, analyze the matching relationship between data patterns and domain ontology models, and automatically generate mapping recommendations to optimize entity relationships in static and dynamic knowledge graphs. S1.4 Based on the optimized static and dynamic knowledge graphs, efficient integration and computation across data sources are achieved through distributed data querying; S1.

5. Through data quality assessment, data lineage tracing, and data security control, the accuracy and security of data at each stage are ensured in real time, providing reliable support for the entire data processing chain.

8. The automated data modeling and optimization system integrating knowledge graphs and ChatBI according to claim 7, characterized in that, The parsing layer specifically performs the following operations: S2.

1. Using graph embedding and text embedding techniques, entities and relationships in static and dynamic knowledge graphs are transformed into vector representations. Entity vectors integrate structural features and descriptive text information, while relationship vectors reflect semantic associations. S2.

2. A division of labor model is adopted between a large language model and a lightweight dedicated model to provide model capability support for natural language processing and SQL generation. The large language model is responsible for natural language understanding and SQL framework generation, while the lightweight dedicated model is responsible for data prediction and execution path optimization. S2.3 Based on the vector representation generated in step S2.1 and the large language model in step S2.2, an enhanced retrieval generation system is constructed by integrating a vector database. The system stores entity and relation vectors of the knowledge graph and historical SQL statement vectors in the vector database to achieve efficient retrieval. The retrieval information is integrated to enhance the context, improve the semantic understanding accuracy of the large language model for user questions, and lay the foundation for subsequent SQL generation. S2.4 Based on the vector representation generated in step S2.1, the division of labor mode in step S2.2, and the user intent parsing results output in step S2.3, and with the constraints of static and dynamic knowledge graphs, the pre-trained natural language to SQL conversion model is used to automatically generate SQL query statements that conform to business logic. S2.5 Define a triggering mechanism based on user query frequency, data change events, or abnormal business indicators. Combine the time-series awareness model to optimize the vector representation of the incremental data in step S2.1, ensuring that the dynamic knowledge graph is synchronized with business changes in real time. The optimization results are also synchronously fed back to the retrieval enhancement generation system in step S2.3 and the SQL query statement generation stage in step S2.4, forming a dynamic closed loop.

9. The automated data modeling and optimization system integrating knowledge graphs and ChatBI according to claim 8, characterized in that, The large language model is a GPT model or an LLaMA model with the ability to deeply understand natural language and generate structured content. It is configured to: receive natural language instructions input by the user, parse the semantic information and task objectives in the instructions, and generate structured content that conforms to grammatical rules and logical structure based on the domain knowledge obtained through pre-training. The lightweight dedicated model is a miniaturized model designed for data prediction or path optimization tasks. It is configured to: receive the structured content output by the large language model as input; in the data prediction scenario, calculate the feature parameters of the input based on domain-specific training data and output the prediction result; in the path optimization scenario, perform algorithmic iterative optimization on the initial execution path of the input to simplify the process; and through collaboration with the large language model, form a technical link from natural language instruction parsing to efficient execution of the dedicated task, thereby improving the accuracy and efficiency of task processing.

10. The automated data modeling and optimization system integrating knowledge graphs and ChatBI according to claim 8, characterized in that, The application layer specifically performs the following operations: S3.1 provides a natural language interactive interface, supporting users to ask questions about report data in the form of voice or text, and realize real-time data interaction; S3.2 Based on the semantic content of the user query, combined with the entity relationship types of the knowledge graph, and referring to the pre-established entity relationship type and chart type mapping table, automatically match and recommend suitable visualization charts. S3.3 Based on the recommendation results, the SQL query results are directly converted into diverse visualization charts, and the display method is optimized by combining the semantic information of static and dynamic knowledge graphs to highlight key data issues; S3.4 Based on the generated charts and related data, and relying on the semantic reasoning capabilities of static and dynamic knowledge graphs, the system performs root cause analysis on data anomalies and provides intelligent early warnings through path search and causal relationship inference, helping users quickly locate the root cause of problems and avoid potential risks in advance.

Images