Natural language data query method and system based on intent recognition

Abstract

This invention discloses a natural language data query method and system based on intent recognition, belonging to the field of data processing technology. The method includes: receiving a user's natural language query request; semantically encoding the query statement using a BERT pre-trained model; inputting the semantic vector into an intent classifier to identify the query intent type; extracting query entities based on the intent type using a named entity recognition model; mapping the extracted entities to an enterprise semantic layer; generating a structured query statement based on the intent type and the mapped fields; executing the query and returning a visual result. The core innovations of this invention lie in employing an intent-entity joint recognition mechanism, an adaptive time parsing engine, a context-aware dialogue state machine, a dynamic adaptation mechanism for the enterprise semantic layer, a multi-dimensional visualization recommendation algorithm, and an incremental learning optimization mechanism. The intent recognition accuracy reaches over 92% (compared to 75% for traditional methods), the query response time is 1.8 seconds (compared to 10 seconds for traditional methods), the multi-turn dialogue success rate is 89% (compared to 45% for traditional methods), and the user satisfaction rate is 95% (compared to 65% for traditional methods).

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a natural language data query method and system based on intent recognition. This invention belongs to the field of business intelligence technology, specifically involving natural language processing (NLP), intent recognition, named entity recognition (NER), semantic mapping, and visualization generation, and can be widely applied to scenarios such as enterprise data analysis, business decision support, and intelligent report generation. Background Technology

[0002] As enterprises deepen their digital transformation, data has become a core asset driving business decisions. However, traditional data analysis tools (such as BI tools and reporting systems) typically require users to have SQL knowledge or be proficient in complex drag-and-drop interfaces, which poses a significant technical barrier for business personnel without a technical background. Statistics show that over 75% of business personnel are unable to independently explore and analyze data due to this technical barrier, severely hindering the efficiency of data-driven decision-making within enterprises.

[0003] In recent years, the rapid development of natural language processing technology has given rise to Natural Language Query (NLQ) technology, allowing users to obtain data insights by asking questions in natural language. Existing NLQ systems mainly suffer from the following technical shortcomings: Technical Deficiency 1: Separation of Intent Recognition and Entity Extraction. Existing technologies typically employ a sequential processing flow of "first recognizing the intent, then extracting the entity," with the intent recognition and entity extraction modules operating independently and lacking an effective coordination mechanism. This separate architecture results in limited accuracy in intent recognition (typically 70-78%) and an inability to dynamically adjust the entity extraction strategy based on the intent type. When a user queries "the top five products by sales volume in the last three months," if the system fails to accurately recognize the "ranking" intent, the entity extraction module cannot obtain the key entity "top five," leading to query failure.

[0004] Technical Deficiency Two: Lack of Adaptability in Time Parsing. Current technologies typically use a fixed default time window (e.g., the last 30 days) for relative time terms such as "recent" and "recently," failing to dynamically adjust based on business scenarios and product lifecycle characteristics. For the FMCG industry, "recently" should be understood as 7-14 days; while for the durable goods industry, "recently" should be understood as 30-90 days. This fixed time parsing strategy results in a large number of queries having time ranges that do not meet actual business needs, with time parsing accuracy typically only reaching 60-70%.

[0005] Technical Deficiency 3: Rigid and Monotonous Enterprise Semantic Layer. Existing systems typically employ a single, static dictionary mapping mechanism, directly mapping business terms to database fields. This single mapping method cannot adapt to the differentiated terminology habits of different industries, departments, and users. For example, "sales revenue" is called "operating income" in the finance department and "transaction amount" in the sales department; different companies may even have custom names. The static mapping mechanism limits the accuracy of semantic mapping (typically 80-85%), requiring users to learn and memorize the system's fixed terminology.

[0006] Technical Deficiency 4: Lack of Context Awareness. Existing technologies are mostly single-turn query systems, unable to handle multiple follow-up questions and abbreviated expressions. When a user asks "sales ranking," and the system returns the result, if the user follows up with "the top three" or "look by region," the existing system cannot understand the relationship between the follow-up question and the preceding text. It cannot interpret "top three" as referring to "the top three ranked products," or "by region" as referring to adding a region dimension for grouping. This lack of context awareness results in an extremely low success rate for multi-turn dialogues (typically 40-50%), severely limiting the user experience.

[0007] Technical Deficiency 5: Lack of Personalization in Visual Recommendations. Existing systems typically recommend charts based on simple rule matching (e.g., recommending pie charts for single dimensions, and bar charts for two dimensions), without comprehensively considering multi-dimensional factors such as data characteristics, analysis scenarios, user preferences, and contextual continuity. The recommended chart types often do not match the user's actual analytical needs, requiring the user to manually adjust the chart type, thus reducing analytical efficiency.

[0008] Technical Deficiency Six: Lack of Adaptive Learning Capability. The existing system's performance is fixed after deployment and cannot self-optimize based on user feedback. As business changes and user habits evolve, the system's intent recognition accuracy, semantic mapping accuracy, and other metrics will gradually decline, requiring manual retraining and deployment, resulting in high maintenance costs and delayed response times.

[0009] Technical Deficiency Seven: Insufficient System Response Performance. Existing technologies suffer from excessively long response times (typically 5-15 seconds) when processing complex queries, primarily due to the lack of effective caching mechanisms and query optimization strategies. The BERT model has a high computational overhead (approximately 500ms-1s per inference), and frequent model calls lead to a degraded user experience. Summary of the Invention

[0010] To address the aforementioned shortcomings of existing technologies, this invention provides a natural language data query method and system based on intent recognition. Through an intent-entity joint recognition mechanism, an adaptive temporal parsing engine, a three-layer architecture of enterprise semantic layer, a context-aware dialogue state machine, a multi-dimensional visualization recommendation algorithm, and an incremental learning optimization mechanism, this invention solves problems in existing technologies such as the separation of intent recognition and entity extraction, lack of adaptability in temporal parsing, rigid semantic mapping, lack of context awareness, lack of personalization in visualization recommendations, lack of adaptive learning capabilities, and insufficient response performance.

[0011] The first technical problem solved by this invention: collaborative optimization of intent recognition and entity extraction. This invention proposes an intent-entity joint recognition mechanism that dynamically feeds back the intent recognition results to the entity extraction module, achieving collaborative optimization between the two. Specifically, after the intent classifier outputs the intent type and confidence level, the system dynamically adjusts the extraction weights of various entities based on the intent type. For example, when the intent is identified as a "trend query," the weight of the time entity is increased to 0.5; when it is identified as a "comparison query," the weight of the dimension entity is increased to 0.5. This dynamic weight adjustment mechanism allows the entity extraction module to focus on the entity types most relevant to the intent, significantly improving the overall query success rate. Technical results: Intent recognition accuracy increased from 75% to 92%, entity extraction accuracy increased from 78% to 94%, and overall query success rate increased from 52% to 88.5%.

[0012] The second technical problem solved by this invention: Adaptive optimization of time analysis This invention proposes an adaptive time parsing engine that dynamically determines the time range of relative time terms based on product lifecycle types (FMCG, durable goods, seasonal goods). The engine first identifies the product category in the query (through entity extraction or user profiling), and then loads the corresponding time rules based on the product category's lifecycle characteristics. For example, for FMCG, "recent" corresponds to 7 days and "recently" corresponds to 14 days; for durable goods, "recent" corresponds to 30 days and "recently" corresponds to 90 days. This adaptive parsing strategy enables time parsing to accurately match actual business needs. Technical results: Time parsing accuracy increased from 65% to 91%, significantly improving the accuracy of the time dimension in queries.

[0013] The third technical problem solved by this invention: dynamic adaptation of the enterprise semantic layer. This invention proposes a three-layer semantic layer architecture for enterprises (basic dictionary layer + department adaptation layer + personalized extension layer) to achieve dynamic adaptation of terminology mapping. The basic dictionary layer stores industry-standard terminology mappings (e.g., "sales revenue → revenue"), the department adaptation layer stores specialized terminology for different business departments (e.g., "operating revenue → revenue" for the finance department), and the personalized extension layer stores user-defined terminology (e.g., "GMV → revenue" for a specific company). The three-layer dictionary uses a priority overlay mechanism (personalized > department > basic), dynamically loading the corresponding business dictionary based on the user's department. This three-layer architecture enables the system to accurately adapt to the terminology habits of different users. Technical results: Semantic mapping accuracy improved from 84% to 96%, and user learning costs reduced by 80%.

[0014] The fourth technical problem solved by this invention: context-aware multi-turn dialogue This invention proposes a context-aware dialogue state machine that supports multi-turn follow-up questions and referential resolution. The system maintains a dialogue state object, saving the current dialogue's topic entity, selected dimensions, selected metrics, selected filters, and query history. When a user asks a follow-up question, the system first parses the relationship between the follow-up question and the context, identifies referential words in the follow-up question (such as "this," "this category," "top five"), traces back through the dialogue history stack to find the specific entity corresponding to the referential word, replaces the referential word with the specific entity, and finally completes the omitted query elements (such as time, dimension, metric) based on context information. Technical effects: The success rate of multi-turn dialogues is increased from 45% to 89%, supporting an average of 5 consecutive rounds of follow-up questions, significantly improving the user experience in complex analysis scenarios.

[0015] The fifth technical problem solved by this invention: multi-dimensional visual intelligent recommendation This invention proposes a multi-dimensional weighted scoring algorithm that comprehensively considers four dimensions—data matching degree, intent adaptation degree, user preference degree, and contextual consistency—to intelligently recommend chart types. The algorithm formula is: Score(Chart_i) = α·Data Matching Degree_i + β·Intent Adaptability_i + γ·User Preference_i + δ·Contextual Consistency_i, where α + β + γ + δ = 1, α = 0.35, β = 0.30, γ = 0.25, and δ = 0.10. Data matching degree evaluates the degree of matching between the chart type and data features (e.g., matching time series data with a line chart); intent adaptation degree evaluates the degree of matching between the chart type and the query intent (e.g., matching a trend query with a line chart); user preference degree provides personalized recommendations based on the user's historical choices; and contextual consistency ensures that the recommendation results remain consistent with the continuity of the conversation. Technical effects: Visual recommendation accuracy increased from 58% to 87%, and the frequency of users manually adjusting charts decreased by 78%.

[0016] The sixth technical problem solved by this invention: Incremental learning and adaptive optimization This invention proposes an incremental learning optimization mechanism to continuously optimize system performance based on user feedback. The system collects three types of feedback: explicit feedback (users actively mark "this is not what I'm looking for"), implicit feedback (users switch chart types), and indirect feedback (users export data and share results). When the accumulated feedback data reaches 100 pieces, an incremental learning process is triggered: freezing the BERT pre-trained layer, fine-tuning the classification head layer for 3 epochs, and dynamically adjusting the visualization recommendation weights. This incremental learning mechanism enables the system to adapt to business changes and evolving user habits. Technical results: Three months after the system went live, the intent recognition accuracy increased from 88.2% to 92.1%, and the chart recommendation acceptance rate increased from 72.5% to 85.7%.

[0017] The seventh technical problem solved by this invention: optimization of system response performance This invention significantly improves system response performance through a three-level caching architecture and query optimization strategies. The three-level cache includes: L1 query result cache (caching complete query results, TTL 1 hour), L2 semantic vector cache (caching BERT encoded results, TTL 24 hours), and L3 entity cache (caching entity retrieval results, TTL 30 minutes). When the cache is hit, the result is returned directly without re-executing BERT inference and SQL generation. Simultaneously, the system automatically optimizes the generated SQL, including index recommendation, predicate pushdown, and partition pruning techniques. Technical results: query response time is reduced from 10 seconds to 1.8 seconds, cache hit rate reaches 62%, and database CPU utilization is reduced by 45%. Attached Figure Description

[0018] Figure 1 This diagram illustrates the layered architecture of a natural language data query method and system based on intent recognition. It showcases the user interaction layer, API gateway layer, intelligent processing layer, and support service layer, along with the data flow and interaction relationships between each layer. The diagram explains how the system receives user natural language queries, utilizes intelligent processing modules such as BERT semantic encoding, intent recognition, entity extraction, and semantic mapping, and ultimately generates visualized results.

[0019] Figure 2 This diagram illustrates the intent recognition process of a natural language data query method and system based on intent recognition. It details the five-category intent classification system (ranking query, trend query, comparison query, percentage query, and detail query) and the technical mechanism for dynamically adjusting entity extraction weights based on intent type. The diagram explains how intent recognition results are fed back to the entity extraction module, enabling collaborative optimization between the two.

[0020] Figure 3This is a flowchart illustrating the entity extraction process of a natural language data query method and system based on intent recognition. It showcases the parallel extraction process for four types of entities (time entities, dimensional entities, metric entities, and filter entities), and the technical principle behind the adaptive time parsing engine dynamically determining the relative time range based on product lifecycle type. The diagram explains how named entity recognition is achieved using the BiLSTM-CRF model, combined with time rule bases for time standardization.

[0021] Figure 4 This is a semantic mapping flowchart for a natural language data query method and system based on intent recognition. It illustrates the three-layer architecture design of the basic dictionary layer, department adaptation layer, and personalized extension layer, as well as the priority coverage mechanism between each layer. The diagram explains how business terms are mapped to database fields level by level, achieving dynamic adaptation and accurate conversion of terms.

[0022] Figure 5 This is a flowchart illustrating the multi-turn dialogue context management process of a natural language data query method and system based on intent recognition. It showcases the state transition logic of the dialogue state machine, including the technical processes of context saving, follow-up questioning and relatedness identification, reference resolution, element completion, and loop return. The diagram explains how the system supports multi-turn follow-up questions and reference resolution, enabling context-aware continuous dialogue.

[0023] Figure 6 This is a flowchart of the overall method and system for natural language data query based on intent recognition. It shows the complete technical process from receiving natural language queries to returning visualized results, including key steps such as semantic encoding, PCA dimensionality reduction, intent recognition, entity extraction, semantic mapping, SQL generation, query execution, and result visualization, as well as the data dependencies between each step.

[0024] Figure 7 The chart compares the results of this invention with traditional techniques using bar charts and line graphs. The comparison spans nine key technical dimensions (intent recognition accuracy, entity extraction accuracy, time parsing accuracy, semantic mapping accuracy, multi-turn dialogue success rate, visual recommendation accuracy, overall query success rate, average response time, and user satisfaction rating). The chart also includes statistical significance verification results from one-way ANOVA. This demonstrates that the present invention represents a statistically significant technological advancement compared to existing technologies. Detailed Implementation

[0025] Example 1: System Overall Architecture Reference Figure 1 The overall architecture of the system of this invention includes four layers: user interaction layer, API gateway layer, intelligent processing layer and support service layer.

[0026] The user interaction layer is responsible for receiving user input and displaying query results. This layer supports three access methods: Web interface (React / Vue front-end framework), mobile APP (iOS / Android native or cross-platform framework), and API interface (RESTful API for third-party system integration). Users input query requests in natural language (such as "the top five mobile phone brands in East China in terms of sales volume in the last three months"). The interaction layer sends the request to the API gateway layer and receives the returned visual results for rendering and display.

[0027] The API gateway layer is responsible for unified access, authentication, routing, and rate limiting of requests. This layer implements the OAuth 2.0 authentication mechanism to verify user identity and permissions; it implements intelligent routing, distributing requests to the corresponding processing services based on request type; it implements traffic control, using a token bucket algorithm to limit requests per second (default 1000 QPS) to prevent system overload; and it implements request logging, recording information such as request source, processing time, and response status for monitoring and auditing.

[0028] The intelligent processing layer is the core of the system, implementing the core technical logic for natural language querying. This layer comprises eight core modules: BERT semantic encoding module, intent recognition module, entity extraction module, semantic mapping module, SQL generation module, query execution module, visualization generation module, and context management module. These modules communicate asynchronously via message queues (RabbitMQ / Kafka) to ensure system stability under high concurrency scenarios. This layer also implements a three-level caching architecture (L1 query result cache, L2 semantic vector cache, and L3 entity cache), significantly improving system response performance.

[0029] The support service layer provides underlying service support for the intelligent processing layer. This layer includes a context management service (managing multi-turn dialogue states and implementing referential resolution), a caching service (using a Redis cluster to implement a three-level cache), an incremental learning service (collecting user feedback and triggering model updates), a database connection pool (managing database connections and implementing read-write separation), and a log monitoring service (recording system logs and enabling real-time monitoring). The support service layer adopts a microservice architecture, with each service deployed and scaled independently to ensure high availability and scalability of the system.

[0030] Example 2: Joint Processing of Intent Recognition and Entity Extraction Reference Figure 2 This embodiment details the joint processing flow of intent recognition and entity extraction. The flow includes the following steps: Step S1: Semantic Encoding and Dimensionality Reduction. The system receives the user's natural language query "Top five mobile phone brands in sales revenue over the last three months" and first performs semantic encoding using a BERT pre-trained model. The BERT model uses the Chinese BERT-base version (12-layer Transformer, 768 hidden layer dimensions), encoding the input sentence into a 768-dimensional semantic vector. To reduce the complexity of subsequent calculations, Principal Component Analysis (PCA) is used to reduce the 768-dimensional vector to 256 dimensions. PCA dimensionality reduction retains more than 95% of the variance information while reducing the vector dimension by 66%, significantly improving the computational efficiency of subsequent intent classification. The dimensionality-reduced semantic vector is denoted as V∈R^256.

[0031] Step S2: Intent Classification. The dimensionality-reduced semantic vector V is input into a five-class intent classifier. The classifier uses a Multilayer Perceptron (MLP) structure: Input layer 256-dimensional → Hidden layer 1 (512 neurons, ReLU activation) → Hidden layer 2 (256 neurons, ReLU activation) → Output layer 5-dimensional (Softmax activation). The output layer corresponds to five intent types: ranking query (output node 0), trend query (output node 1), comparison query (output node 2), percentage query (output node 3), and detail query (output node 4). The classifier outputs the probability distribution of each intent type, and the intent type corresponding to the maximum probability is taken as the recognition result. In this example, the classifier outputs a probability of 0.87 for ranking query, 0.08 for trend query, 0.03 for comparison query, 0.01 for percentage query, and 0.01 for detail query, therefore it is identified as "ranking query".

[0032] Step S3: Dynamic Weight Adjustment. Based on the identified intent type "ranking query", the system dynamically adjusts the extraction weights of various entities. Specific weight settings are as follows: Dimension entity weight 0.4, Measure entity weight 0.4, Filter entity weight 0.2, Time entity weight 0.3. Compared to the default weights (0.25 each), the ranking query increases the weights of Dimension and Measure entities because the core elements of a ranking query are "by which dimension" and "by which metric".

[0033] Step S4: Entity Extraction. The Named Entity Recognition (NER) model is used to extract the query entities. The NER model employs a BiLSTM-CRF architecture: Input layer: BERT encoding (768 dimensions) + character-level features (100 dimensions) → BiLSTM layer (256-dimensional hidden layer, bidirectional) → CRF layer (Conditional Random Field, modeling label dependencies) → Output layer (BIO tags: B-time, I-time, B-dimension, I-dimension, B-metric, I-metric, B-filter, I-filter, O). In this example, the NER model extracts the following entities: time entity "last three months", dimension entity "brand", metric entity "sales revenue", filter entity "mobile phone", and ranking entity "top five".

[0034] Step S5: User Confirmation and Looping. The system displays the identified intent type and extracted entities to the user, asking, "You want to query: the top five mobile phone brands by sales revenue over the past three months, is that correct?" If the user confirms, the system proceeds to the semantic mapping step; if the user denies or requests a modification (e.g., "No, I want to see the top ten"), the system adjusts the weights and re-extracts the entities. In this example, the user confirms that everything is correct, and the system proceeds to the next step.

[0035] Technical effectiveness verification: On a dataset from a large retail enterprise, the intent recognition accuracy of this embodiment reached 92.3%, and the entity extraction accuracy reached 94.1%. Compared with the traditional separate architecture (intent recognition 75%, entity extraction 78%), the overall query success rate increased from 52% to 88.4%.

[0036] Example 3: Entity Extraction and Adaptive Temporal Parsing Reference Figure 3 This embodiment details the entity extraction process and the technical principles of adaptive temporal parsing. Entity extraction includes the parallel identification of four types of entities: temporal entities, dimensional entities, metric entities, and filtered entities.

[0037] Time entity recognition employs a dual-strategy mechanism of rule matching and model prediction. For absolute times (such as "January 2024", "Q4 2023"), regular expression rules are used for direct matching and standardization; for relative times (such as "recent", "recently", "last year", etc.), an adaptive time parsing engine is used. The engine first identifies the product category in the query (through user profiles or the product entities mentioned above), and then loads the corresponding time rules based on the product category's lifecycle characteristics.

[0038] Taking fast-moving consumer goods (such as mobile phones) as an example, if the current date is March 20, 2024, and a user queries "recently," the engine loads the fast-moving consumer goods rule ("recently" = 7 days) and resolves it to a time range [2024-03-13, 2024-03-20]. If the user queries "last year's period," the engine resolves it to [2023-03-01, 2023-03-31] (same month as last year). For seasonal products (such as air conditioners), since the current month is March, "recently" may correspond to the previous quarter (last year's Q4) or the remaining days of the current quarter, and the engine intelligently determines this based on the context.

[0039] Dimensional entity recognition. Dimensional entities refer to the grouping dimensions involved in the query, such as "brand" in "view by brand" and "region" in "sales status in each region". Dimensional recognition adopts a combined strategy of dictionary matching and semantic similarity calculation. The system maintains an enterprise dimension dictionary (such as brand, region, channel, product line, etc.). Dimensions existing in the dictionary are directly matched; for dimensions outside the dictionary (such as "view by major region"), the semantic similarity with known dimensions is calculated (e.g., "major region" and "region" have a similarity of 0.85). If the similarity exceeds the threshold (0.80), normalization mapping is performed.

[0040] Entity identification. Entities refer to the quantitative metrics involved in the query, such as "sales revenue," "order volume," and "profit margin." Entity identification also employs a strategy of dictionary matching combined with semantic similarity calculation. The system maintains an enterprise metric dictionary, supporting the identification of composite metrics (e.g., identifying "sales revenue year-on-year growth rate" as measuring "sales revenue" + calculating "year-on-year" + aggregating "growth rate"). For aggregation methods (e.g., "sum," "average," "maximum"), the system automatically identifies them through keyword matching or semantic understanding.

[0041] Entity identification during filtering. Entities used in the query refer to filtering conditions such as "mobile phone category," "East China region," and "price greater than 1000." Filtering identification employs a pattern matching + semantic parsing strategy. The system supports three filtering modes: equality filtering ("mobile phone category" → category='mobile phone'), range filtering ("price greater than 1000" → price>1000), and fuzzy filtering ("best-selling products" → top 20% of sales). Filtering conditions support AND / OR logical combinations, and the system identifies logical relationships using connectives ("AND," "OR").

[0042] Technical performance verification: On a dataset from an e-commerce company, the accuracy rate of time parsing reached 91.2% (compared to 65.4% for traditional fixed rules), the accuracy rate of dimension recognition reached 93.5%, the accuracy rate of metric recognition reached 95.1%, and the accuracy rate of filtering condition recognition reached 89.7%.

[0043] Example 4: Enterprise Semantic Layer Mapping Refer to Figure 4 , this embodiment details the three - layer architecture design and dynamic mapping mechanism of the enterprise semantic layer.

[0044] The basic dictionary layer stores the mapping from industry - common terms to standard fields, such as "sales amount → revenue", "order quantity → order_count", "number of customers → customer_count". The basic dictionary adopts a standardized industry - term system (such as e - commerce industry standard terms) to ensure the system has cross - industry general capabilities. The basic dictionary is maintained by the system administrator and updated with new industry terms regularly.

[0045] The department adaptation layer stores the mapping of professional terms for different business departments. Taking the sales department as an example, the mapping includes "transaction amount → revenue", "number of orders → order_count", "customer → customer"; taking the finance department as an example, the mapping includes "operating income → revenue", "number of order transactions → order_count", "number of users → customer_count". The department dictionary is dynamically loaded according to the user's affiliated department. When a user logs in from the sales department, the sales - department dictionary is loaded; when a user logs in from the finance department, the finance - department dictionary is loaded.

[0046] The personalized extension layer stores the mapping of user - defined terms. Users can add personal habitual expressions in the system settings, such as "GMV → revenue", "UV → visitor_count". The personalized dictionary is only valid for the current user and does not affect other users. Users can also mark "common terms", and the system preferentially matches the marked terms.

[0047] Priority override mechanism. The three - layer dictionary adopts a priority override mechanism, with the priority from high to low being: personalized extension layer (priority 3) > department adaptation layer (priority 2) > basic dictionary layer (priority 1). When there are different mappings of the same term in multiple dictionaries, the mapping of the high - priority dictionary overrides that of the low - priority dictionary. For example, the basic dictionary maps "sales amount" to "revenue", but a certain user maps "sales amount" to "gmv" in the personalized dictionary, then "sales amount" in the user's query is preferentially mapped to "gmv".

[0048] Fuzzy matching and error correction. To improve the user experience, the system supports fuzzy matching and spelling correction. When the edit distance between the term entered by the user and the term in the dictionary ≤ 2 (for example, the edit distance between "sales received amount" and "sales amount" is 1), the system automatically corrects and prompts the user "Do you want to query'sales amount'?". The system also supports pinyin fuzzy matching (such as "xiaoshoue" matches "sales amount").

[0049] Technical effectiveness verification: In a multi-department collaborative enterprise test, the semantic mapping accuracy reached 96.3% (compared to 84.5% for traditional single mapping), the success rate of user queries increased from 65% to 91%, and the user learning cost was reduced by 80%.

[0050] Example 5: Multi-turn Dialogue and Context Management Reference Figure 5 This embodiment details the context management process and reference resolution mechanism for multi-turn dialogues.

[0051] Dialogue state is saved. After each round of dialogue, the system saves the current dialogue state to the dialogue history stack. Dialogue state objects include: subject entity (e.g., "sales revenue"), selected dimensions (e.g., ["brand"]), selected metrics (e.g., ["sales revenue"]), selected filters (e.g., {"category":"mobile phone"}), query history (a complete list of query statements), and time range (e.g., "last three months"). Dialogue state is saved to a Redis cache with a TTL of 30 minutes, supporting session recovery.

[0052] Follow-up question identification. When a user enters a new query, the system first determines whether it is a follow-up question. Follow-up question identification rules include: query length ≤ 10 characters (e.g., "the top three"), containing pronouns ("this", "this category"), and containing omitted expressions ("East China region", lacking a subject). If any of these rules are met, the system identifies it as a follow-up question and loads the previous dialogue state.

[0053] Referential resolution. For follow-up questions containing referential terms, the system traces back through the dialogue history stack to find the specific entity corresponding to the referential term. Referential terms like "this" or "this type" typically refer to the aforementioned subject entity or result set; the system traces back to find the most recent subject entity (e.g., "mobile phone"). Referential terms like "top five" or "top ten" refer to the range of results in the aforementioned query; the system traces back to find the most recent ranking condition. The system replaces the referential term with the specific entity, such as completing "the top three" as "which brands are ranked in the top three?".

[0054] Element completion. Follow-up questions often omit some query elements (such as time, dimensions, and metrics). The system completes the omitted elements based on the context. Completion rules include: if the follow-up question does not specify a time, it inherits the time range from the preceding text; if the follow-up question does not specify a dimension but includes "by XX", then XX is added as a dimension; if the follow-up question does not specify a metric, it inherits the metric from the preceding text. For example, if the preceding query is "ranking of mobile phone sales in the last three months", and the follow-up question is "view by channel", the system completes it as "view mobile phone sales by channel in the last three months".

[0055] Example of a multi-turn dialogue. A complete 4-turn dialogue example is as follows: Technical effectiveness verification: In a test at a large enterprise, the success rate of multi-turn dialogues reached 89.2% (compared to 45.3% for traditional single-turn systems), with an average of 4.7 dialogue rounds, and user satisfaction improved from 3.2 / 5.0 to 4.6 / 5.0.

[0056] Example 6: Generation of Visualized Results Reference Figure 6 This embodiment details the steps for generating visualization results in the overall method flow.

[0057] Data Feature Analysis. The system first analyzes the data features of the query results, including: number of dimensions (1-dimensional, 2-dimensional, 3+-dimensional), number of measures (single measure, multiple measures), data type (numerical, categorical, time-based), and data volume (number of records <100, 100-1000, >1000). These data features determine the range of selectable chart types.

[0058] Multi-dimensional weighted scoring. The system uses a multi-dimensional weighted scoring algorithm to intelligently recommend chart types. The scoring formula is: Score(Chart_i) = 0.35·Data matching degree_i + 0.30·Intent fit degree_i + 0.25·User preference degree_i + 0.10·Context consistency_i.

[0059] Data matching assesses the degree of match between chart type and data characteristics. Rules include: Time series data + trend query → Line chart matching score 0.95; Single dimension + percentage query → Pie chart matching score 0.90; Two dimensions + comparison query → Bar chart matching score 0.88; Multi-dimensional + detail query → Table matching score 0.92.

[0060] The intent fit assessment evaluates the degree of match between chart type and query intent. Ranking queries fit bar charts (0.90) and column charts (0.85); trend queries fit line charts (0.95) and area charts (0.80); comparison queries fit bar charts (0.90) and grouped bar charts (0.88); percentage queries fit pie charts (0.90) and donut charts (0.85); and detail queries fit tables (0.95) and cards (0.75).

[0061] User preference is used to personalize recommendations based on users' historical choices. The system records the frequency of users' selection of each chart type over the past 30 days. If a user selects a line chart 8 times out of the past 10 trend queries, then the user preference score for line charts is 0.80. The user preference score weight of 0.25 ensures personalized recommendations without over-reliance on historical behavior.

[0062] Contextual consistency ensures that the recommended results remain consistent with the continuity of the conversation. If the previous query used a bar chart, the contextual consistency score for recommending a bar chart in the current query will improve by 0.15; if the user manually switched the chart type in the previous query, the contextual consistency score for recommending the type switched by the user in the current query will improve by 0.20.

[0063] Chart Generation and Rendering. The system selects the highest-scoring chart type and generates charts using visualization libraries such as ECharts / AntV. The charts include full interactive features: hovering over data points displays detailed values, legend filtering is supported, zooming and panning are supported, and data export (CSV / Excel) is supported. The system also generates natural language summaries ("This month's sales increased by 15% month-on-month, with East China contributing the most") to help users understand data insights.

[0064] Technical effectiveness verification: In a test among a group of enterprise users, the accuracy of visual recommendation reached 87.3% (compared to 58.1% for traditional rule matching), the frequency of users manually adjusting charts decreased by 78%, and the first recommendation acceptance rate increased by 42%.

[0065] Example 7: Incremental Learning and Adaptive Optimization Reference Figure 6 This embodiment details the technical principles and implementation process of incremental learning in the incremental learning module.

[0066] User feedback is collected. The system collects three types of user feedback: explicit feedback (users actively click buttons such as "This is not what I'm looking for" or "Chart type is inappropriate"), implicit feedback (users manually change the chart type or adjust the query conditions after submitting the recommended results), and indirect feedback (users export data, share results, or save queries, indicating that the query results are valuable). Feedback data is stored in the feedback database, recording the feedback type, feedback time, user ID, original query, original intent recognition result, original recommended chart, and the result corrected by the user.

[0067] Triggering conditions are determined. The system scans the feedback database daily. When a user's feedback data accumulates to 100 entries, the incremental learning process is triggered. Triggering conditions also include: the intent recognition accuracy rate dropping by more than 3% for 7 consecutive days, and the user actively reporting "the system's understanding ability has decreased" more than 5 times.

[0068] Model fine-tuning. Incremental learning employs a model fine-tuning strategy, freezing the BERT pre-trained layers (to avoid catastrophic forgetting and retain general language understanding capabilities) and fine-tuning only the classification head layers (intent classification MLP and entity extraction BiLSTM-CRF). Fine-tuning uses collected feedback data as the training set, with the learning rate set to 1 / 10 of the original learning rate (to prevent overfitting), and training for 3 epochs.

[0069] Visual recommendation weight adjustment. Incremental learning also dynamically adjusts the weight coefficients of visual recommendations. If a user frequently switches the recommended chart type, it indicates that the recommendation is inaccurate, and the system reduces the default weight and increases the weight of user preference. For example, if a user frequently switches the recommended grouped bar chart to a stacked bar chart, the system reduces the default weight of the grouped bar chart from 0.30 to 0.20 and increases the weight of the stacked bar chart to 0.35.

[0070] A / B Testing and Deployment. The fine-tuned model is first A / B tested with a small user group (5% of users) to compare the accuracy and user satisfaction of the new and old models. If the new model outperforms the old model (accuracy improvement >2% or satisfaction improvement >0.2 points), it is then deployed to all users.

[0071] Technical Performance Verification: Table 4 shows the incremental learning performance of the system three months after its launch. In the first month, the intent recognition accuracy was 88.2%, the chart recommendation acceptance rate was 72.5%, and user satisfaction was 4.2 / 5.0. In the second month, the accuracy improved to 90.5%, the acceptance rate was 78.3%, and the satisfaction rate was 4.4 / 5.0. In the third month, the accuracy was 92.1%, the acceptance rate was 85.7%, and the satisfaction rate was 4.6 / 5.0. Incremental learning enabled continuous improvement in system performance, adapting to changes in business needs and evolving user habits.

[0072] Example 8: Comparative Experiment Verification Reference Figure 7 This embodiment verifies the significant technological advancements of the present invention compared to existing technologies through comparative experiments.

[0073] Experimental Setup. The experimental dataset was collected from three months of real business query logs of a large e-commerce company, totaling 15,000 query statements, covering five intent types (ranking, trend, comparison, percentage, and details), involving 10 business dimensions and eight metrics. The control group used an existing natural language query system (BERT encoding + 5-category intent recognition + static entity extraction, without intent-entity joint recognition, adaptive temporal parsing, and three-layer semantic mapping). The experimental group adopted the complete technical solution of this invention.

[0074] Experimental Results. The comparison results of the nine key technical indicators are shown in Table 5. Intent recognition accuracy: Control group 75.3%±3.2%, experimental group 92.0%±1.8%, an improvement of 16.7 percentage points, p<0.001 (highly significant); Entity extraction accuracy: Control group 78.1%±2.9%, experimental group 94.0%±1.5%, an improvement of 15.9 percentage points, p<0.001; Temporal parsing accuracy: Control group 65.2%±4.1%, experimental group 91.0%±2.1%, an improvement of 25.8 percentage points, p<0.001; Semantic mapping accuracy: Control group 84.5%±2.3%, experimental group 96.0%±1.2%, an improvement of 11.5 percentage points, p<0.001; Multi-turn dialogue success rate: Control group 45.0%±5.5%. The overall success rate of the query was 52.0%±3.5%, while that of the experimental group was 88.5%±1.6%, an increase of 36.5 percentage points, p<0.001; the average response time was 10.2±2.1 seconds in the control group and 1.8±0.3 seconds in the experimental group, a decrease of 82.4%, p<0.001; and the user satisfaction score was 3.2±0.5 / 5.0 in the control group and 4.6±0.2 / 5.0 in the experimental group, an increase of 44%, p<0.001.

[0075] One-way ANOVA results: The F-statistics for all key indicators were greater than the critical value (F>4.0), and the p-values ​​were all less than 0.001, indicating that there was a highly significant difference between the experimental group and the control group (*** indicates highly significant). The technical solution of this invention has a statistically significant technological advancement compared with the prior art.

[0076] Industrial application value demonstration Market application prospects The Business Intelligence (BI) market continues to expand, with the global BI market size projected to reach $30 billion by 2025, representing a compound annual growth rate (CAGR) of 12.5%. The Chinese market is growing even faster, with a CAGR of 18.3%. Natural language querying, as an innovative direction in the BI field, currently has a market penetration rate of less than 10%, indicating huge growth potential. This invention can be applied to the following scenarios: enterprise data analysis platforms (ERP / CRM / SCM system integration), e-commerce operation analysis systems, financial risk control data analysis, medical and health data querying, and government data open platforms. Potential customers include: Fortune 500 companies (approximately 500), large and medium-sized Chinese enterprises (approximately 500,000), and SaaS service providers (approximately 5,000). Conservatively estimated, the market size for this invention's technical solution could reach 5 billion RMB per year.

[0077] Technology replicability and scalability The technical solution of this invention possesses excellent replicability and scalability. The solution adopts a modular architecture, with each module (semantic encoding, intent recognition, entity extraction, and semantic mapping) interacting through standard API interfaces, allowing for independent deployment and replacement. Core models such as the BERT model and the BiLSTM-CRF model are implemented using standard deep learning frameworks (PyTorch / TensorFlow), easily migrated to different hardware platforms (CPU / GPU / TPU). The three-layer architecture of the enterprise semantic layer enables the system to quickly adapt to different industries and enterprises: the basic dictionary layer provides general capabilities, the departmental adaptation layer allows for rapid customization via configuration files, and the personalized extension layer supports user-defined expansion. The system supports horizontal scaling; processing capacity can be increased by adding service nodes. A single node supports 100 QPS, and a 10-node cluster can support 1000 QPS, meeting the high-concurrency needs of large enterprises.

[0078] Economic Benefit Analysis Direct economic benefits: Assuming a medium-sized enterprise (1000 employees) deploys this invention's system, the annual direct economic benefits include: a reduction of 5 IT developers (saving 750,000 RMB / year in labor costs), a reduction in report development cycle from an average of 3 days to 5 minutes (saving 1.2 million RMB / year in labor costs), and an increase in the proportion of business personnel conducting independent analysis from 20% to 85% (estimated business value improvement of 2 million RMB / year due to data-driven decision-making). The total annual economic benefits are approximately 3.95 million RMB, with a return on investment (ROI) of approximately 1:8.

[0079] Indirect economic benefits: The proportion of data-driven decision-making increased from 45% to 78%, significantly improving the quality of decision-making; business personnel can explore data without learning SQL, and the data literacy rate increased from 15% to 65%; the data analysis cycle was shortened from days to minutes, and the market response speed increased tenfold.

[0080] Social benefits: The social benefits of this invention include lowering the technical threshold for enterprise data analysis, promoting data democratization, enabling business personnel to easily access data insights; improving the data literacy of the public and cultivating a corporate culture of data-driven decision-making; and accelerating the digital transformation process of enterprises.

Claims

1. A natural language data query method based on intent recognition, characterized in that, Includes the following steps: S1: Receives natural language query requests from users; S2: The query statement is semantically encoded using the BERT pre-trained model to generate a 768-dimensional semantic vector, and the principal component analysis method is used to reduce the 768-dimensional semantic vector to 256 dimensions to obtain the dimensionality-reduced semantic vector. S3: Input the reduced semantic vector into a five-class intent classifier to identify the query intent type as ranking query, trend query, comparison query, percentage query, or detail query. S4: Based on the intent type, dynamically adjust the extraction weights of various entities and use a named entity recognition model to extract query entities; the query entities include time entities, dimension entities, metric entities, and filter entities; the time entity recognition adopts an adaptive time parsing engine to dynamically determine the time range of relative time words according to the product lifecycle type; S5: Map the extracted entities to the enterprise semantic layer, which includes a basic dictionary layer, a department adaptation layer, and a personalized extension layer. Dynamically load the corresponding business dictionary according to the user's department and convert business terms into database fields. S6: Generate a structured query statement based on the intent type and the mapped fields; S7: Execute the query and return the visualization results; the visualization results are generated using a multi-dimensional weighted scoring algorithm to intelligently recommend chart types.

2. The method according to claim 1, characterized in that, The dynamic adjustment of extraction weights for various entities in step S4 includes: 0.4 for dimension entities, 0.4 for metric entities, and 0.2 for filter entities in ranking queries; 0.5 for time entities, 0.4 for metric entities, and 0.1 for dimension entities in trend queries; 0.5 for dimension entities, 0.4 for metric entities, and 0.1 for dimension entities in comparison queries; 0.4 for dimension entities, 0.1 for time entities in comparison queries; 0.4 for dimension entities, 0.4 for metric entities, and 0.2 for filter entities in percentage queries; and 0.6 for filter entities, 0.2 for time entities, and 0.2 for dimension entities in detail queries.

3. The method according to claim 1, characterized in that, The adaptive time parsing engine dynamically determines the time range of relative time terms based on the product lifecycle type, including: for fast-moving consumer goods, "recent" corresponds to 7 days and "recent" corresponds to 14 days; for durable goods, "recent" corresponds to 30 days and "recent" corresponds to 90 days; for seasonal goods, "recent" corresponds to 14 days or the remaining days of the current quarter.

4. The method according to claim 1, characterized in that, The enterprise semantic layer mapping in step S5 includes: a basic dictionary layer storing the mapping from industry-wide common terms to standard fields; a department adaptation layer storing the mapping of professional terms from different business departments; and a personalized extension layer storing the mapping of user-defined terms. The three-layer dictionary adopts a priority overlay mechanism, with the priority from high to low being the personalized extension layer, the department adaptation layer, and the basic dictionary layer.

5. The method according to claim 1, characterized in that, It also includes multi-turn dialogue context processing steps: saving the topic entity of the current dialogue, selected dimensions, selected metrics, selected filters and query history; when receiving follow-up questions from users, parsing the relationship between the follow-up questions and the context, identifying pronouns in the follow-up questions and performing referential resolution; Based on the aforementioned relationships, omissions in the query elements are completed, and a complete query statement containing contextual information is generated.

6. The method according to claim 5, characterized in that, The referential resolution includes: identifying referential words such as "this", "this category", and "top five" in follow-up questions; tracing back through the dialogue history stack to find the most recent relevant entity and determine the specific entity corresponding to the referential word; replacing the referential word with the determined entity to generate a complete query.

7. The method according to claim 1, characterized in that, The multi-dimensional weighted scoring algorithm in step S7 is as follows: Score(Chart_i) = α·data matching degree_i + β·intent fit degree_i + γ·user preference degree_i + δ·context consistency_i, where α+β+γ+δ=1, α=0.35, β=0.30, γ=0.25, and δ=0.

10. The scores for each dimension are calculated comprehensively based on data characteristics, analysis scenarios, user historical choices, and dialogue continuity, and the chart type with the highest score is returned as the recommendation result.

8. The method according to any one of claims 1-7, characterized in that, It also includes incremental learning optimization steps: collecting user feedback on intent recognition results and chart recommendations, including explicit feedback, implicit feedback and indirect feedback; when the accumulated feedback data reaches 100, triggering batch model updates, including freezing the BERT pre-trained layers, fine-tuning the classification head layer for 3 epochs, and dynamically adjusting the visualization recommendation weights; and continuously optimizing the accuracy of intent recognition and chart recommendation based on user feedback.

9. A natural language data query system based on intent recognition, characterized in that, include: The query receiving module is used to receive users' natural language query requests; The semantic encoding module is used to semantically encode the query statement using the BERT pre-trained model, generating a 768-dimensional semantic vector; The intent recognition module is used to input the semantic vector into a five-class intent classifier to identify the query intent type; The entity extraction module is used to dynamically adjust the extraction weights of various entities based on the intent type and to extract the query entities using a named entity recognition model. The semantic mapping module is used to map the extracted entities to the enterprise semantic layer, which includes a basic dictionary layer, a department adaptation layer, and a personalized extension layer. The query generation module is used to generate structured query statements based on the intent type and the mapped fields; the execution module is used to execute the query; and the visualization module is used to intelligently recommend chart types and return visualization results using a multi-dimensional weighted scoring algorithm.

10. The system according to claim 9, characterized in that, Also includes: The context management module is used to save context information for multi-turn dialogues, and supports continuous follow-up questions and reference resolution. The caching module is used to cache query results using a three-level caching architecture, prioritizing the return of cached data for the same query. The learning module is used to optimize intent recognition and chart recommendation algorithms based on user feedback.

Images