A conversational data query method based on composite knowledge units and heterogeneous indexes

Abstract

The application relates to the technical field of natural language processing, and in particular relates to a conversational data query method based on a composite knowledge unit and a heterogeneous index. The method comprises the following steps: accessing a multi-source heterogeneous data system, encapsulating data entities with business semantic association as a composite knowledge unit; constructing a semantic vector index and a structure heat index based on access frequency for the composite knowledge unit, and dynamically weighting and fusing to generate a heterogeneous joint index; analyzing a user query based on a conversation context, generating a semantic representation in combination with a buffer state, searching and generating a query plan containing an interpretable Token by using the heterogeneous joint index; finally, querying heterogeneous data sources in parallel, fusing results to generate a natural language answer, and updating the structure heat index weight by reinforcement learning through user implicit feedback. Through the composite knowledge unit and the self-evolution heterogeneous index, the application realizes accurate and efficient query of multi-source heterogeneous data, and improves the accuracy and adaptability of the conversational data query.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically, to a conversational data query method based on composite knowledge units and heterogeneous indexes. Background Technology

[0002] With the deep integration of artificial intelligence and big data technologies, enterprise-level data exhibits characteristics of multi-source and heterogeneity, with structured and unstructured data coexisting and distributed across different data systems. Conversational data query, as a convenient human-computer interaction method, has been widely applied in scenarios such as intelligent customer service and business intelligence analysis. Its core requirement is to allow users to quickly and accurately obtain integrated information from cross-data sources through multi-turn dialogues in natural language, without needing to master professional query syntax or data storage structures. Currently, conversational data query technology typically needs to complete core steps such as data access, query parsing, index retrieval, and answer generation. Among these, the effective organization of data entities, the efficient construction of index structures, and the accurate utilization of conversational context are key factors in improving query efficiency and accuracy. Existing technologies typically perform preliminary integration of multi-source heterogeneous data, construct a single type of index for query retrieval, and implement multi-turn dialogue support through simple context caching to meet basic query needs. Meanwhile, the development of semantic retrieval technology provides technical support for conversational queries. By vectorizing text information through pre-trained language models, semantic similarity-based retrieval is achieved, while traditional structured indexes optimize retrieval paths based on field access frequency. Both types of indexes have been widely used in their respective fields.

[0003] However, existing conversational data query methods still have many shortcomings in practical applications, making it difficult to meet the high-precision, high-efficiency, and evolvable query requirements of enterprise-level scenarios. First, existing technologies mostly employ a single index structure, either relying solely on semantic vector indexes, which suffer from high retrieval latency and resource consumption due to the "curse of dimensionality" in high-dimensional vectors, and struggle to consider the structural characteristics of the data; or relying solely on traditional structure indexes, which cannot effectively capture the business semantic relationships of the data, leading to deviations in query intent parsing, especially in multi-turn dialogues where issues such as unresolved referencing and inaccurate omissions and completions are prone to occur. Second, existing data organization methods do not consider the business semantic dependencies between data entities, and do not uniformly encapsulate entities with relationships, resulting in complex data relationship logic and cumbersome query path planning when querying across data sources. Furthermore, they cannot effectively integrate the quality information of structured data with the semantic information of unstructured data, affecting the completeness and reliability of the answer. Furthermore, existing methods lack an effective self-evolution mechanism. Fixed index weights prevent dynamic optimization based on implicit feedback from user query behavior, leading to persistent difficulties in improving query efficiency and accuracy over long-term use. Additionally, the lack of interpretable tokens in query plans prevents users from tracing the rationale behind routing decisions, reducing the credibility of query results. Moreover, the imperfect mechanism for storing multi-turn dialogue states fails to effectively support constraint completion in subsequent queries, further impacting query accuracy.

[0004] Therefore, developing a session-based data query method that can effectively integrate multi-source heterogeneous data, construct efficient heterogeneous indexes, accurately utilize session context, and achieve self-evolution has become a technical challenge that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a conversational data query method based on composite knowledge units and heterogeneous indexes, in order to address the above-mentioned deficiencies of the prior art.

[0006] The technical solution adopted by this invention to solve its technical problem is: A conversational data query method based on composite knowledge units and heterogeneous indexes includes the following steps: The steps for constructing a composite knowledge unit are as follows: access a multi-source heterogeneous data system, parse data entities and their relationships, and encapsulate data entities with business semantic relationships into composite knowledge units; the composite knowledge unit includes structural slots, text blocks, operation interfaces, a pre-set logic layer, and a buffer for temporarily storing multi-turn dialogue states; Heterogeneous index construction steps: Construct a four-dimensional heterogeneous vector index system for the composite knowledge unit, including a knowledge subject vector index, a knowledge keyword vector index, a form metadata index, and a structure popularity index. Dynamically adjust the fusion weight of the four-dimensional heterogeneous vector index system according to the query context through an attention mechanism to generate a heterogeneous joint index. The structure popularity index is constructed based on the access frequency of fields in historical query logs. Query parsing and routing steps: Based on the session context, the user's current query is parsed, and combined with the dialogue state temporarily stored in the buffer, a semantic representation containing the query intent and key elements is generated; the heterogeneous composite index is used to perform dual-granularity retrieval and multi-path recall fusion, locate the target composite knowledge unit and the internal query path of the heterogeneous composite index, and generate a query plan containing interpretable tokens. Answer generation and evolution steps: According to the query plan, the large model calls the pre-set logic layer to generate execution instructions; parallel queries are performed on heterogeneous data sources, and the query results are merged to generate natural language answers and fed back to the user; implicit user feedback on the answers is collected, and the weights of the structure popularity index are updated through reinforcement learning mechanism using the implicit feedback.

[0007] Preferably, the composite knowledge unit construction step further includes: By collecting metadata and performing field-level lineage analysis, the business logic dependencies between data entities are analyzed, and the boundaries of composite knowledge units are determined. The construction of composite knowledge units adopts a three-layer encapsulation, including: Basic information layer: Structured fields and data quality tags are used as structure slots, unstructured text and anchor links between unstructured text and structure slots are used as text blocks, and a list of associated knowledge IDs is added to form a graph-like association network. Pre-defined logic layer: encapsulates executable code logic, including retrieval rules, data processing logic, and result verification logic, and sets natural language trigger conditions; Search adaptation layer: Configures weighted search keywords, which cover domain terms and contextual vocabulary to provide a basis for multi-dimensional search; The pre-defined query templates are used as operation interfaces and are collectively encapsulated into composite knowledge units.

[0008] Preferably, the heterogeneous index construction step further includes: Knowledge Subject Vector Index: Vectorizes the basic information layer text of composite knowledge units to generate vectors representing the overall semantics; Knowledge Keyword Vector Index: Extracts the configuration keywords of the retrieval adaptation layer and generates a weighted vector set for accurate matching of domain terms; Form metadata index: It is split into a field information vector library and a field unique value vector library to avoid the interference of unique value retrieval with field sorting; and a table for table and field association information is added to store primary keys, foreign keys and business relationships. Structure popularity index: Based on historical query logs, it statistically analyzes the frequency of field access and the popularity of paths; By dynamically adjusting the fusion weights of the knowledge subject vector index, knowledge keyword vector index, form metadata index, and structure popularity index based on the query context through an attention mechanism, a heterogeneous composite index is generated.

[0009] Preferably, the query parsing and routing steps further include: The system performs referential resolution and omission completion on the user's current query, and combines the dialogue state temporarily stored in the buffer to generate a semantic representation containing the query intent and key elements; it also utilizes a heterogeneous composite index for dual-granularity retrieval. Calculate the similarity between the complete semantic vector of the user query and the knowledge subject vector; Calculate the weighted similarity between the extracted query keyword vector and the knowledge keyword vector; Synchronously trigger form metadata retrieval, and use the improved Reverse Ranking fusion algorithm (RRF) to fuse multi-way retrieval results to determine the optimal field access path; A query plan is generated based on the optimal field access path. The query plan includes cross-unit connection logic, data source switching instructions, preset logic trigger flags, and an interpretability token that records the reason for the routing decision.

[0010] Preferably, the answer generation and evolution steps further include: Based on the query plan, the large model calls the pre-set logic layer to generate execution instructions. The code executor identifies the pre-set logic trigger flags in the instructions and directly executes the encapsulated retrieval, calculation, and verification logic, without requiring the large model to autonomously derive business rules. The federated query engine initiates parallel queries to heterogeneous data sources to obtain structured result sets and unstructured text fragments. The query results from different sources are merged according to the association logic of composite knowledge units and sorted by relevance or time. The fused results are converted into natural language answers using a text generation model, and an implicit feedback collection interface is embedded in the answers. Implicit feedback signals are generated based on subsequent actions to the answer, including clicking, asking follow-up questions, or making corrections; the weights of corresponding fields in the structural heat index are adjusted using reinforcement learning algorithms to optimize routing decisions for subsequent queries.

[0011] Preferably, the buffer is used to store the specific values ​​or conditions of composite knowledge units involved in multi-turn dialogues, for subsequent query parsing to automatically complete constraints.

[0012] Preferably, the data quality labels include field completeness and timeliness scores, which are used to evaluate the reliability of the results when querying routes.

[0013] Preferably, the knowledge subject vector index and the knowledge keyword vector index are collectively referred to as the multi-voic semantic vector index. The multi-voic semantic vector index is generated based on a pre-trained language model, and the structural popularity index is implemented based on an inverted index or a B-tree.

[0014] The beneficial effects of this invention are as follows: 1. This application encapsulates entities with business semantic relationships in multi-source heterogeneous data through "composite knowledge units," integrating structure slots, text blocks, and pre-built operation interfaces into reusable knowledge modules. This encapsulation breaks through the limitations of traditional "data structure-oriented" queries, enabling the system to directly understand business themes such as "customer orders" and "product sales," rather than processing table fields in isolation. For example, data quality tags (completeness and timeliness scores) in structure slots can prioritize highly reliable data sources during query routing, while anchor links between text blocks and structured data (such as "price XXX yuan" associated with the "product price" slot) ensure accurate binding between unstructured semantics and structured attributes, ultimately achieving a high degree of alignment between query intent and business logic, reducing manual adaptation costs.

[0015] 2. This application supports access to heterogeneous data from multiple sources, including relational databases (MySQL), NoSQL (MongoDB), file systems (CSV / JSON), and API interfaces. Through metadata collection and field lineage analysis, it parses business logic dependencies and reorganizes scattered data into composite knowledge units based on business themes. During queries, the federated query engine accesses each data source in parallel according to the optimal path of the heterogeneous composite index, automatically handling syntax differences and type conversions, avoiding the cumbersome nature of traditional ETL processes. For example, when simultaneously querying structured order data for the "Customer" unit and unstructured descriptive text for the "Product" unit, the system can directly fuse the results through the association logic of composite knowledge units, significantly improving the efficiency of collaborative utilization of multi-source data.

[0016] 3. This application innovatively constructs a "multimodal semantic vector index" (capturing the deep semantics of text blocks and slot names) and a "structural popularity index" (based on the field access frequency and path popularity of historical logs), and dynamically weights and merges them according to the query context (e.g., fuzzy queries focus on semantics, while precise queries focus on popularity). The two-stage retrieval mechanism (first using the semantic index to recall candidate units, and then using the popularity index to prune and optimize the path) balances semantic relevance and query efficiency: for example, when a user performs a fuzzy query for "high-end mobile phone sales", the semantic index can recall the "product sales" unit, while the popularity index prioritizes the path of frequently accessed fields such as "sales amount" and "category", thus improving the positioning accuracy and response speed compared to a single index.

[0017] 4. This application embeds "explainable tokens" into the query plan to record key reasons for routing decisions, such as "selecting composite knowledge unit A due to semantic similarity of 0.92," "using path P1 due to its historical popularity ranking of 1st," and "structure slot X data quality score of 9.5." These tokens are presented in a structured form (such as appended to query logs or answer notes), allowing users to clearly understand "why this data source was selected" and "why this path was taken," thus solving the trust problem of traditional black-box queries. For example, when users question the results, they can trace back to "data quality labels" or "historical successful paths" through the tokens, enhancing the credibility of the interaction.

[0018] 5. This application generates implicit feedback signals by collecting subsequent user actions (clicks, follow-up questions, corrections), and dynamically adjusts the field / path weights of the structural popularity index using reinforcement learning (such as Q-learning). For example, if a user clicks on the query results for "Brand A" multiple times, the system will automatically increase the popularity weight of query paths related to "Brand A," ensuring that subsequent similar queries prioritize this efficient path. This closed-loop mechanism of "query-feedback-evolution" enables the system to adapt to changes in data distribution (such as emerging popular fields) and shifts in user preferences, continuously optimizing routing decisions without manual intervention and maintaining leading query performance in the long term.

[0019] 6. This application temporarily stores specific values ​​(e.g., "Region = East China"), confirmed conditions (e.g., "Time = 2023Q4"), and user preferences involving complex knowledge units in multi-turn dialogues through a "dialogue state buffer." Combined with referential resolution (e.g., associating "it" with "this order" in the previous round) and omission completion (e.g., completing "check sales revenue" as "check sales revenue of XX product"), fragmented queries are expanded into complete semantic expressions. For example, after a user specifies "Region → Category → Time" in three rounds, the buffer automatically integrates the constraints, avoiding repetitive input and improving the contextual coherence of multi-turn dialogues compared to traditional conversational systems. This is particularly suitable for complex query scenarios where requirements need to be clarified step by step.

[0020] 7. This application encapsulates unstructured text (such as product descriptions and user reviews) into "text blocks" and links them with structured slots via anchor points. When merging query results, the two types of data are integrated according to the association logic of composite knowledge units. For example, when answering "high-end mobile phone recommendations," the system can simultaneously return structured fields (brand, price, sales volume) and unstructured text blocks (user review summaries), and sort them by relevance. This ensures that the answer has both data accuracy and semantic richness, meeting diverse needs from "data statistics" to "trend interpretation" and expanding the application boundaries of conversational queries. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort: Figure 1 This is a schematic diagram illustrating the steps of a conversational data query method based on knowledge units and heterogeneous indexes in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of the three-layer structure of the composite knowledge unit in an embodiment of this application.

[0023] Figure 3 This is a schematic diagram of the four-dimensional heterogeneous index system architecture of an embodiment of this application.

[0024] Figure 4 This is a schematic diagram of the dual-granularity retrieval and RRF fusion process in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, a clear and complete description will be provided below in conjunction with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0026] Preferred embodiments of the present invention, for example Figures 1 to 4 As shown, a conversational data query method based on composite knowledge units and heterogeneous indexes includes the following steps: S1. Composite Knowledge Unit Construction Steps: Integrate with multi-source heterogeneous data systems, parse data entities and their relationships, and encapsulate entities with business semantic relationships into composite knowledge units. Composite knowledge units include structural slots, text blocks, operation interfaces, pre-defined logic layers, and buffers for temporarily storing multi-turn dialogue states. These buffers are used to store specific values ​​or conditions related to composite knowledge units in multi-turn dialogues, allowing subsequent queries to automatically complete constraints. Specifically, this involves integrating multi-source heterogeneous data systems, including but not limited to relational databases (such as MySQL, PostgreSQL), NoSQL databases (such as MongoDB, Redis), file systems (such as CSV, JSON, XML), and API interfaces. Metadata information (such as table structure, field definitions, and data types) from each data source is obtained through metadata collection tools (such as Apache Atlas, Amundsen). Field lineage analysis tools (such as Apache Falcon, OpenLineage) are used to parse the business logic dependencies and data flow between data entities (such as tables, views, files, and API return objects). Based on this, the boundaries of composite knowledge units are determined, i.e., which entities are grouped together due to shared business themes or frequent collaborative participation in the same type of query.

[0027] S11. Through metadata collection and field lineage analysis, analyze the business logic dependencies between data entities to determine the boundaries of composite knowledge units; construct composite knowledge units using a three-layer encapsulation, including: The basic information layer uses structured fields and data quality tags as structured slots, and unstructured text and anchor links between unstructured text and structured slots as text blocks. A list of associated knowledge IDs is added to form a graph-like association network. The pre-defined logic layer encapsulates executable code logic, including retrieval rules, data processing logic, and result verification logic, and sets natural language trigger conditions. The retrieval adaptation layer configures weighted retrieval keywords, which cover domain terminology and contextual vocabulary to provide a basis for multi-dimensional retrieval.

[0028] S12. Structured data segments and data quality are used as structure slots, unstructured text and anchor links between unstructured text and structure slots are used as text blocks, and pre-defined query templates are used as operation interfaces, all encapsulated into composite knowledge units. Data quality tags include field completeness and timeliness scores, used to evaluate the reliability of results during query routing. Structure slots are used to hold structured data. Each structure slot corresponds to one or more structured fields, containing basic information such as field name, data type, value range, and whether it can be nullable. In particular, predefined data quality tags are used as important attributes of structure slots. Data quality tags include, but are not limited to, field completeness (e.g., missing value ratio), timeliness (e.g., data update delay), accuracy (e.g., cross-validation results with other trusted sources), and consistency (e.g., consistency of cross-table field value logic) scores. These scores are used to evaluate the likelihood of results during query routing, prioritizing high-quality data sources. Text blocks are used to hold unstructured or semi-structured text data, such as product descriptions, user reviews, and news summaries. Simultaneously, anchor links are established between unstructured text and structured slots. For example, in a product description text, if it mentions "the price of this product is -- yuan," then the position "-- yuan" will be anchored to the corresponding "product price" structure slot, realizing the association between text and structured data. The operation interface is a pre-set set of query templates, designed for common query intents supported by this composite knowledge unit. For example, for the "customer orders" composite knowledge unit, templates such as "query the total amount of a customer's orders in the last N days" and "query the list of orders with order status X" can be pre-set. The templates contain placeholders, which are used to receive parameters from user queries or the dialogue state buffer. The dialogue state buffer is used to temporarily store the specific values, confirmed conditions, user preferences, and other information designed for this composite knowledge unit in multiple rounds of dialogue. For example, constraints such as "region = East China" and "product category = electronic products" gradually specified by the user in multiple rounds of dialogue will be temporarily stored here for automatic completion and reference during subsequent query parsing.

[0029] S13. Registration and Management of Composite Knowledge Units: Register the completed composite knowledge units and their metadata (such as unique identifier, name, description, included structural slots, text block summaries, operation interface lists, etc.) to the knowledge unit repository for centralized management and section control.

[0030] S2. Heterogeneous Index Construction Steps: Semantic vector indexes and structural popularity indexes are constructed separately for each composite knowledge unit. These two indexes are then dynamically weighted and merged to generate a heterogeneous composite index. The structural popularity index is constructed based on the access frequency of fields in historical query logs. To improve the efficiency and accuracy of query retrieval, a heterogeneous composite index is constructed and maintained for each composite knowledge unit.

[0031] S21. Vectorize the business meaning of the text block, structural slot name, and enumeration value of the composite knowledge unit. For example, the structural slot name "customer satisfaction rating" and its possible enumeration values ​​(such as "high", "medium", and "low") will be converted into vectors respectively; generate multimodal semantic vectors and construct a multimodal semantic vector index.

[0032] S22. Collect historical query logs, statistically analyze the access frequency and query path of each field, and generate a structural popularity index. Generating the structural popularity index includes data collection, popularity calculation, and index construction. Data collection primarily involves collecting historical query logs, which should include query time, user ID, query statement, involved composite knowledge unit ID, specific fields accessed, query path (e.g., entry point, related queries), and the number of query results. Popularity calculation mainly statistically analyzes the access frequency of structural slots within each composite knowledge unit (e.g., total access count, average daily access count), query path usage frequency, and average response time. A popularity calculation formula can be defined according to business needs. For example, after dimensionless or normalized processing of each parameter (e.g., access frequency, reciprocal of average response time, result click-through rate, etc.), a weighted sum is performed to obtain a dimensionless popularity value for comparing the "popularity" of different paths. Index construction is based on the statistically obtained popularity information to build a structural popularity index. Inverted indexes can be used to record frequently accessed fields and paths, or B-trees and other structures can be used to store hot values ​​for fast retrieval and updates.

[0033] S23. A heterogeneous composite index is generated by dynamically adjusting the fusion weights of the multimodal semantic vector index and the structural popularity index based on the query context using an attention mechanism. The fusion weight strategy employs an attention mechanism to dynamically adjust the weight contributions of the multimodal semantic vector index and the structural popularity index during the retrieval process based on the current query context (such as query keywords, user historical preferences, and dialogue stage). For example, the weight of the semantic vector index can be increased for fuzzy or exploratory queries; the weight of the structural popularity index can be increased for precise queries or queries with known popular fields. Furthermore, the heterogeneous composite index is not a physically single index, but a logically fused retrieval mechanism. In actual retrieval, the two indexes can be invoked simultaneously or sequentially based on dynamic weights, and the results can be comprehensively sorted and filtered. For example, the semantic vector index is first used to recall the Top-K candidate composite knowledge units, and then the structural popularity index is used to score and prune the query paths within these candidate units.

[0034] S3. Query Parsing and Routing Steps: Understand user intent, locate target knowledge units, and plan query paths. Based on the session context, parse the user's current query and combine it with the temporarily stored dialogue state in the buffer to generate a semantic representation containing query intent and key elements. Use a heterogeneous composite index to retrieve the semantic representation, locate the target composite knowledge unit and the internal query path of the heterogeneous composite index, and generate a query plan containing interpretable tokens.

[0035] S31. Perform pronoun resolution and omission completion on the user's current query, and combine it with the dialogue state temporarily stored in the buffer (such as the query conditions confirmed in the previous round and the entities the user is interested in) to generate a complete semantic representation. Pronoun resolution and omission completion involve performing Natural Language Understanding (NLU) on the user's current query, focusing on handling pronoun references (such as "it," "the order") and omitted components (such as "query sales revenue" needing to be completed based on the preceding text to "query sales revenue of product XX"). Semantic representation involves transforming the parsed complete semantic expression into a structured set of query intents (such as "query," "statistics," "comparison") and key elements (such as entities, attributes, conditions, aggregation functions, sorting methods, etc.), forming a semantic representation vector (which can be further vectorized for easier matching with the index).

[0036] S32. Input the key elements into the heterogeneous joint index, and first use the multimodal semantic vector index to recall a list of candidate composite knowledge units that are semantically similar to the query intent and key elements. This process can utilize vector similarity (such as cosine similarity).

[0037] The structural heat index is then used to prune the query paths within candidate units to determine the optimal field access path. For recalled candidate composite knowledge units, the structural heat index is used to evaluate and rank their possible query paths. Paths with high access frequency, fast response speed, and high historical success rate are prioritized. For complex queries that may involve the association of multiple composite knowledge units, the structural heat index can also help determine the rationality and cost of the association. Finally, the optimal field access path (including the composite knowledge units involved, the connection relationships, and the specific fields) is determined.

[0038] Multi-voic semantic vector index generates dense vectors based on pre-trained language models, which is good at capturing semantic relevance; structural heat index is implemented based on inverted index or B-tree, which is good at quickly finding high-frequency elements and hot paths.

[0039] S33. Generate a query plan based on the optimal field access path. The query plan includes, but is not limited to, cross-unit connection logic (if the query involves multiple composite knowledge units, the relationship between the multiple composite knowledge units must be specified), data source switching instructions (specifying which heterogeneous data sources need to be obtained from), query operators (such as filtering, projection, aggregation, sorting, grouping, etc.), and interpretable tokens that record the reasons for the routing decision (recording the key reasons for the routing decision, such as "choosing composite knowledge unit A because it has the highest semantic similarity score (0.92)", "using path P1 because it ranks first in historical access popularity", "the data quality score of structural slot X is 9.5 (out of 10)". These tokens will be embedded in the final query plan to enhance interpretability.

[0040] S4. Answer generation and evolution steps: According to the query plan, query heterogeneous data sources in parallel, merge the query results to generate natural language answers and feed them back to the user; collect implicit feedback from users on the answers, and use the implicit feedback to update the weight of the structure popularity index through a reinforcement learning mechanism.

[0041] S41. Based on the query plan, the large model calls the pre-defined logic layer to generate execution instructions. The code executor identifies the pre-defined logic trigger flags in the instructions and directly executes the encapsulated retrieval, calculation, and verification logic without requiring the large model to autonomously deduce business rules. Parallel queries are initiated to the heterogeneous data sources involved through the federated query engine. The federated query engine is responsible for decomposing the same query plan into sub-queries for different data sources and handling syntax differences, data type conversions, and result merging between different data sources. Structured results (such as tabular data) and unstructured text fragments are obtained.

[0042] S42. Merge query results from different sources according to the association logic of composite knowledge units. For example, associate the name from the "Customer" unit and the order details from the "Order" unit through the customer ID. The merged results can be sorted by relevance (such as the degree of matching with the query intent), time (such as the latest data taking priority), importance (such as the preset business priority).

[0043] S43. Use a text generation model to convert the fused and sorted results into fluent and easy-to-understand natural language answers, and embed implicit feedback collection interfaces in the answers. For example, provide clickable lightweight feedback components such as "Is it correct?" or "Is it satisfactory?" at key information points, or guide users to provide feedback through specific interaction methods.

[0044] S44. Generate implicit feedback signals based on subsequent actions taken in response to the answer. These actions may include clicking, asking follow-up questions, or making corrections. Monitor subsequent actions taken by the user in response to the answer and convert them into implicit feedback signals. For example: Clicking: A user clicks on a link or recommended content in the answer, which may indicate interest; Asking a new question about the answer: A user may indicate dissatisfaction with parts of the answer or a desire for further understanding; Correction: A user explicitly points out an error in the answer; Dwell time: The length of time a user stays on the answer page; No action: A user quickly leaves, which may indicate disinterest in the answer or dissatisfaction.

[0045] Reinforcement learning algorithms are used to adjust the weights of corresponding fields in the structure popularity index in order to optimize the routing decisions for subsequent queries.

[0046] In practical applications, suppose a user wants to query "In the fourth quarter of 2023, in East China, which brand of high-end smartphones saw the fastest sales growth?" The following is a detailed description of the method steps in Example 1: S1: The system has built a composite knowledge unit for "product sales", which includes structural slots such as "time", "region", "product category", "brand", "sales amount", "sales year-on-year growth rate", as well as related product description text blocks and operation interfaces.

[0047] S2: A semantic vector index (containing the names of each slot and text block vectors) and a structural popularity index (such as "sales revenue" and "growth rate" being accessed frequently) were built for the "Product Sales" unit.

[0048] S3: S31: The user's current query is parsed and, combined with the absence of previous dialogue, yields the semantic representation: Intent = "Query the fastest growing brand", Key Elements = {Time: 2023Q4, Region: East China, Category: High-end smartphones, Metric: Sales growth rate, Sort: Descending, Top 1}.

[0049] S32: By using a heterogeneous composite index and a semantic vector index to recall the "Product Sales" unit, and a structural popularity index to confirm that "Sales Growth Rate" is a high-popularity field, the query path is determined.

[0050] S33: Generate a query plan that includes querying relevant data from the "Product Sales" unit, filtering by conditions, sorting by growth rate to get the Top 1, and recording an interpretable token.

[0051] S4: S41: The federated query engine performs parallel queries on related databases.

[0052] S42: Merge query results and sort by growth rate.

[0053] S43: The text generation model generates the answer: "In the fourth quarter of 2023, among the high-end smartphones in East China, Brand A had the fastest sales growth, with a growth rate of X%." S44: When a user clicks on "Brand A" in the answer, the system uses this as positive implicit feedback and fine-tunes the weight of the query path related to "Brand A" through reinforcement learning.

[0054] Example 2 Taking intelligent querying of regulatory reports in the financial sector as an example, for instance, a bank needs to query "the loan balance and non-performing loan ratio of high-end manufacturing enterprises in East China in the fourth quarter of 2023". The traditional solution requires manual data extraction and calculation across systems, which takes about 2 hours and is prone to errors.

[0055] S1: Constructing composite knowledge units Basic Information Layer: The structure slots include loan balance (quality score 9.2), non-performing loan ratio (9.5), industry classification, and region; the text block is a description of the credit contract; the associated knowledge ID links to the "Customer Information" unit.

[0056] Pre-defined logic layer: Search rules: Time = 2023Q4, Region = East China, Industry = High-end manufacturing; Data processing failure rate = Non-performing loans / Loan balance; Result verification: Return an exception prompt when the loan balance is ≤0.

[0057] Search adaptation layer: Configure keywords: high-end manufacturing industry (weight 0.9), East China region (0.85).

[0058] S2: Constructing a four-dimensional heterogeneous index The knowledge subject vector index stores the overall semantics of "corporate loans"; The knowledge keyword index stores weighted vectors such as high-end manufacturing; the form metadata index splits the field library (name / alias / description) and the unique value library (stored independently); the structure popularity index counts the top 3 most frequently used fields with the worst failure rate.

[0059] S3: Query Parsing and Routing A user asked: "What was the non-performing loan ratio for high-end manufacturing in East China in the fourth quarter of 2023?" Dual-granularity retrieval: long-to-long matching similarity 0.93; short-to-short matching weighted score for high-end manufacturing 0.855.

[0060] After RRF reordering, the "Corporate Loans" unit was located, and a query plan containing an interpretable token was generated: this unit was selected because of its semantic similarity of 0.93; the default rate field was used because it was among the top 3 most popular.

[0061] S4: Answer Generation and Self-Evolution The large model calls the preset logic CALC_BAD_RATE, and the code executor automatically calculates the defect rate as 1.2%.

[0062] Output answer: "The non-performing loan ratio of high-end manufacturing enterprises in East China was 1.2% in the fourth quarter of 2023 (data quality score 9.5)". When a user clicks on the "Failure Rate" detail, the system increases the weight of that field by 5%.

[0063] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A conversational data query method based on composite knowledge units and heterogeneous indexes, characterized in that, Includes the following steps: The steps for constructing a composite knowledge unit are as follows: access a multi-source heterogeneous data system, parse data entities and their relationships, and encapsulate data entities with business semantic relationships into composite knowledge units; the composite knowledge unit includes structural slots, text blocks, operation interfaces, a pre-set logic layer, and a buffer for temporarily storing multi-turn dialogue states; Heterogeneous index construction steps: Construct a four-dimensional heterogeneous vector index system for the composite knowledge unit, including a knowledge subject vector index, a knowledge keyword vector index, a form metadata index, and a structure popularity index. Dynamically adjust the fusion weight of the four-dimensional heterogeneous vector index system according to the query context through an attention mechanism to generate a heterogeneous joint index. The structure popularity index is constructed based on the access frequency of fields in historical query logs. Query parsing and routing steps: Based on the session context, the user's current query is parsed, and combined with the dialogue state temporarily stored in the buffer, a semantic representation containing the query intent and key elements is generated; the heterogeneous composite index is used to perform dual-granularity retrieval and multi-path recall fusion, locate the target composite knowledge unit and the internal query path of the heterogeneous composite index, and generate a query plan containing interpretable tokens. Answer generation and evolution steps: Based on the query plan, the execution instructions are generated by calling the preset logic layer through the large model; Parallel queries are performed on heterogeneous data sources, and the query results are merged to generate natural language answers and then fed back to the user. Collect implicit feedback from users on the answers, and use the implicit feedback to update the weights of the structural heat index through a reinforcement learning mechanism.

2. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 1, characterized in that, The steps for constructing the composite knowledge unit further include: By collecting metadata and performing field-level lineage analysis, the business logic dependencies between data entities are analyzed, and the boundaries of composite knowledge units are determined. The construction of composite knowledge units adopts a three-layer encapsulation, including: Basic information layer: Structured fields and data quality tags are used as structure slots, unstructured text and anchor links between unstructured text and structure slots are used as text blocks, and a list of associated knowledge IDs is added to form a graph-like association network. Pre-defined logic layer: encapsulates executable code logic, including retrieval rules, data processing logic, and result verification logic, and sets natural language trigger conditions; Search adaptation layer: Configures weighted search keywords, which cover domain terms and contextual vocabulary to provide a basis for multi-dimensional search; The pre-defined query templates are used as operation interfaces and are collectively encapsulated into composite knowledge units.

3. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 2, characterized in that, The heterogeneous index construction step further includes: Knowledge Subject Vector Index: Vectorizes the basic information layer text of composite knowledge units to generate vectors representing the overall semantics; Knowledge Keyword Vector Index: Extracts the configuration keywords of the retrieval adaptation layer and generates a weighted vector set for accurate matching of domain terms; Form metadata index: It is split into a field information vector library and a field unique value vector library to avoid the interference of unique value retrieval with field sorting; and a table for table and field association information is added to store primary keys, foreign keys and business relationships. Structure popularity index: Based on historical query logs, it statistically analyzes the frequency of field access and the popularity of paths; By dynamically adjusting the fusion weights of the knowledge subject vector index, knowledge keyword vector index, form metadata index, and structure popularity index based on the query context through an attention mechanism, a heterogeneous composite index is generated.

4. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 1, characterized in that, The query parsing and routing steps further include: The system performs referential resolution and omission completion on the user's current query, and combines the dialogue state temporarily stored in the buffer to generate a semantic representation containing the query intent and key elements; it also utilizes a heterogeneous composite index for dual-granularity retrieval. Calculate the similarity between the complete semantic vector of the user query and the knowledge subject vector; Calculate the weighted similarity between the extracted query keyword vector and the knowledge keyword vector; Synchronously trigger form metadata retrieval, and use the improved Reverse Ranking fusion algorithm (RRF) to fuse multi-way retrieval results to determine the optimal field access path; A query plan is generated based on the optimal field access path. The query plan includes cross-unit connection logic, data source switching instructions, preset logic trigger flags, and an interpretability token that records the reason for the routing decision.

5. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 1, characterized in that, The answer generation and evolution steps further include: Based on the query plan, the large model calls the pre-set logic layer to generate execution instructions. The code executor identifies the pre-set logic trigger flags in the instructions and directly executes the encapsulated retrieval, calculation, and verification logic, without requiring the large model to autonomously derive business rules. The federated query engine initiates parallel queries to heterogeneous data sources to obtain structured result sets and unstructured text fragments. The query results from different sources are merged according to the association logic of composite knowledge units and sorted by relevance or time. The fused results are converted into natural language answers using a text generation model, and an implicit feedback collection interface is embedded in the answers. Implicit feedback signals are generated based on subsequent actions to the answer, including clicking, asking follow-up questions, or making corrections; the weights of corresponding fields in the structural heat index are adjusted using reinforcement learning algorithms to optimize routing decisions for subsequent queries.

6. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 1, characterized in that, The buffer is used to store the specific values ​​or conditions of complex knowledge units involved in multi-turn dialogues, so that subsequent queries can automatically complete the constraints.

7. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 2, characterized in that, The data quality labels include field completeness and timeliness scores, which are used to evaluate the reliability of the results when querying routes.

8. The conversational data query method based on composite knowledge units and heterogeneous indexes according to claim 3, characterized in that, The knowledge subject vector index and the knowledge keyword vector index are collectively referred to as the multi-voic semantic vector index. The multi-voic semantic vector index is generated based on a pre-trained language model, and the structural popularity index is implemented based on an inverted index or a B-tree.

Images