Method and system for performing natural language question and answer directly according to dimension value

Abstract

The invention discloses a method and a system for performing natural language question answering directly according to dimension values. The method comprises the following steps: selecting a corresponding data view according to a subject domain of a user question in a natural language form proposed by a user; performing data replacement on the content of the user question based on data replacement configuration in the data view, and replacing the content of the user question with a standard index or dimension; based on the standard index or dimension, converting a user question into an SQL logical expression through a large model; translating the SQL logical expression into an SQL statement based on configuration information of a semantic model; and executing the SQL statement based on a physical data model, and returning a query result of the SQL statement. According to the invention, the user can use the natural language to carry out interactive dialogue analysis, and the working efficiency and the use experience of the user are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for natural language question answering directly based on dimensional values. Background Technology

[0002] The development of Business Intelligence (BI) has undergone four technological iterations, ultimately achieving a qualitative leap driven by AI-powered large-scale models: from the IT-led, long-cycle data warehouse development of the traditional BI era, to agile BI empowering businesses with self-service analysis capabilities, then lowering the analytical threshold through NL2SQL technology, and finally entering the conversational BI stage centered on LLM—through natural language interaction and deep data processing capabilities, decision-makers can penetrate massive amounts of data in real time, accurately gaining insights in a conversational format, significantly improving the efficiency and accuracy of complex decisions, marking a fundamental leap for BI from the tool level to the cognitive intelligence layer. However, in natural language question answering, it is necessary to clearly specify the conditions for a certain dimension to be a certain value; users need to know what the specific dimension is, making it inconvenient for users to ask questions.

[0003] Prior art 1 (Publication No.: CN117473053A) provides a natural language question answering method, apparatus, medium, and device based on a large language model. The method includes: acquiring a reference knowledge document; segmenting the reference knowledge document into multiple text blocks, and performing word segmentation and word vector representation on the multiple text blocks to obtain a vector database including multiple text feature vectors; acquiring a natural language question input by a user; performing word segmentation and word vector representation on the natural language question to obtain word segmentation vectors; searching the vector database for target text feature vectors whose relevance to the word segmentation vectors is greater than a set threshold; determining the target text block corresponding to the target text feature vector; and performing language processing on the natural language question and the target text block based on a large language model to generate an answer to the natural language question. Prior art 1 requires providing a reference knowledge document, has high knowledge update costs and significant delays, fails to completely eliminate the illusion risk of large language models, and has limited ability to handle complex problems.

[0004] Prior art 2 (Publication No.: CN114896380A) provides a natural language question answering method, system, device, and electronic device. The method includes: classifying user questions to obtain target classification results; based on the target classification results, performing knowledge-based question answering, problem-based question answering, or casual conversation processing on the user questions to obtain question-answering results; and displaying the question-answering results on an interactive interface. Prior art 2 requires preparing a large number of questions, and classification errors can lead to global failure. The application scenarios of this method are limited to solving a specific type of question.

[0005] Existing technology 3 (Publication No.: CN120123365A) provides an intelligent question-asking method and apparatus. Based on a large model and semantic modeling, it includes the following steps: S1, a user interaction module receives natural language questions input by the user; S2, a semantic modeling module collects and integrates relevant data sources; S3, a rule parsing module builds built-in parsing rules; S4, a large model processing module understands and answers the natural language questions input by the user; S5, an SQL conversion module generates general logical SQL based on the natural language questions input by the user through rules and natural language parsing; S6, a data processing and indexing module establishes a data indexing system; S7, a dynamic feedback and learning mechanism is established; S8, a security and privacy protection mechanism is established. Existing technology 3 lacks a mechanism for modifying the content generated by the large model. The content generated by the large model may be incorrect, and it does not consider that the user's question may not be a standard question. Summary of the Invention

[0006] The present invention provides a method and system for performing natural language question answering directly based on dimension values, in order to solve the problem of how to perform accurate natural language question answering directly based on dimension values.

[0007] To address the aforementioned problems, this invention provides a method for natural language question answering directly based on dimension values, the method comprising:

[0008] Select the corresponding data view based on the subject domain of the user's question in natural language format;

[0009] Based on the data replacement configuration in the data view, the content of the user question is replaced with standard indicators or dimensions.

[0010] Based on the aforementioned standard metrics or dimensions, user questions are converted into SQL logical expressions using a large model.

[0011] Based on the configuration information of the semantic model, the SQL logical expression is translated into an SQL statement;

[0012] The SQL statement is executed based on the physical data model, and the query results of the SQL statement are returned.

[0013] Preferably, the step of replacing the user question with a standard metric or dimension based on the data replacement configuration in the data view includes:

[0014] For user questions in natural language format, words in the user questions are replaced based on the data replacement configuration;

[0015] The part-of-speech tag of the replaced user question is segmented, and the user question is replaced with the corresponding standard indicator or dimension according to the part-of-speech tag.

[0016] Preferably, the semantic model includes: standard metrics, dimensions, primary keys, and foreign keys;

[0017] The semantic model can be established using data sources or datasets;

[0018] When the semantic type of the semantic model is a dimension, it includes: enumeration, data time, ordinary time, primary key, and foreign key; when the semantic type of the semantic model is a metric, it includes: aggregation function.

[0019] Preferably, the data view includes: dictionary configuration and data replacement configuration;

[0020] The dictionary configuration is used to configure dictionary items corresponding to dimensions;

[0021] The data view is used to configure the content to be replaced by the part of speech, including: part of speech, replacement word, whether to delete special words, condition words corresponding to special word processing, and part of speech corresponding to special word processing;

[0022] The data replacement configuration is used to convert user business terms or colloquialisms into standard metrics or dimensions.

[0023] Preferably, before translating the SQL logical expression into an SQL statement, the SQL logical expression is further modified:

[0024] Check if the SQL statement is valid;

[0025] When the SQL statement is found to be invalid, semantic correction is performed on the SQL statement; the semantic correction includes: query field correction, query condition correction, grouping condition correction, and query time range correction.

[0026] Based on another aspect of the present invention, the present invention provides a system for natural language question answering directly based on dimension values, the system comprising:

[0027] The selection unit is used to select the corresponding data view based on the subject domain of the user's question in natural language form.

[0028] The replacement unit is used to perform data replacement on the content of the user question based on the data replacement configuration in the data view, replacing the content of the user question with standard indicators or dimensions.

[0029] The conversion unit is used to convert user questions into SQL logical expressions using a large model based on the standard indicators or dimensions.

[0030] The translation unit is used to translate the SQL logical expression into an SQL statement based on the configuration information of the semantic model;

[0031] The execution unit is used to execute the SQL statement based on the physical data model and return the query results of the SQL statement.

[0032] Preferably, the replacement unit is configured to perform data replacement on the user question based on the data replacement configuration in the data view, replacing the user question with a standard indicator or dimension, and is further configured to:

[0033] For user questions in natural language format, words in the user questions are replaced based on the data replacement configuration;

[0034] The part-of-speech tag of the replaced user question is segmented, and the user question is replaced with the corresponding standard indicator or dimension according to the part-of-speech tag.

[0035] Preferably, the semantic model includes: standard metrics, dimensions, primary keys, and foreign keys;

[0036] The semantic model can be established using data sources or datasets;

[0037] When the semantic type of the semantic model is a dimension, it includes: enumeration, data time, ordinary time, primary key, and foreign key; when the semantic type of the semantic model is a metric, it includes: aggregation function.

[0038] Preferably, the data view includes: dictionary configuration and data replacement configuration;

[0039] The dictionary configuration is used to configure dictionary items corresponding to dimensions;

[0040] The data view is used to configure the content to be replaced by the part of speech, including: part of speech, replacement word, whether to delete special words, condition words corresponding to special word processing, and part of speech corresponding to special word processing;

[0041] The data replacement configuration is used to convert user business terms or colloquialisms into standard metrics or dimensions.

[0042] Preferably, the conversion unit is further configured to modify the SQL logical expression before translating it into an SQL statement:

[0043] Check whether the SQL logical expression is valid. If the SQL logical expression is found to be invalid, perform semantic correction on the SQL logical expression. The semantic correction includes: query field correction, query condition correction, grouping condition correction, and query time range correction.

[0044] This invention provides a method and system for natural language question answering directly by dimension values. The method includes: selecting a corresponding data view based on the subject domain of a user's question in natural language form; replacing the content of the user's question with standard indicators or dimensions based on the data replacement configuration in the data view; converting the user's question into an SQL logical expression using a large model based on the standard indicators or dimensions; translating the SQL logical expression into an SQL statement based on the configuration information of the semantic model; executing the SQL statement based on the physical data model and returning the query results of the SQL statement. This invention proposes a method for natural language question answering directly by dimension values. The method uses replacement words configured in the data view to replace user business terms or colloquialisms with standard indicators or dimensions. When processing natural language questions, the words in the question are first replaced according to the replacement word configuration; then, the question is segmented according to parts of speech, and the corresponding indicators or dimensions are found based on the parts of speech, and replaced with a format where a certain dimension has a certain value. This form of question is then passed to the large model, which improves the accuracy of the large model in generating SQL logical expressions, thereby improving the accuracy of natural language question answering. The technical solution of this invention facilitates interactive dialogue analysis using natural language, thereby improving user efficiency and user experience. Attached Figure Description

[0045] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0046] Figure 1 This is a flowchart of a method for natural language question answering directly based on dimension values ​​according to a preferred embodiment of the present invention;

[0047] Figure 2 A flowchart illustrating a method for natural language question answering directly based on dimension values ​​according to a preferred embodiment of the present invention; and

[0048] Figure 3 This is a system architecture diagram for natural language question answering based directly on dimension values ​​according to a preferred embodiment of the present invention. Detailed Implementation

[0049] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0050] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0051] Figure 1 This is a flowchart of a method for natural language question answering directly based on dimension values ​​according to a preferred embodiment of the present invention.

[0052] This invention proposes a method for natural language question answering directly based on dimension values, aiming to facilitate interactive dialogue analysis using natural language and improve user efficiency and experience. This invention configures replacement words according to data views, replacing user business terms or colloquialisms with standard indicators or dimensions. By adding a dictionary corresponding to a dimension on the page or reading it from the data source, setting the part-of-speech tag in the dictionary, and then configuring the replacement words corresponding to the part-of-speech tag, a certain dimension value is replaced with a value for that dimension. When processing natural language questions, the words in the question are first replaced according to the configured replacement words; then the question is segmented according to the part-of-speech tag, and the corresponding indicator or dimension is found based on the part-of-speech tag, and replaced with a value for that dimension. This form of question is then passed to the large model, which improves the accuracy of the large model in generating SQL logical expressions, thereby improving the accuracy of natural language question answering.

[0053] This invention eliminates the need to build and maintain a vector database of external reference knowledge documents. Instead, it directly reads or updates dimensional information from the data source through a dynamically configured dictionary, significantly reducing knowledge update costs and latency. Simultaneously, it effectively mitigates the illusion risk of large language models through standardized question replacement and enhances the ability to handle complex data queries (such as multidimensional analysis). This invention completely avoids the question classification step, eliminating global failures caused by classification errors. Furthermore, it is not limited to specific domains (such as course exercises) but supports a wider range of scenarios through a part-of-speech-driven replacement mechanism, improving flexibility and reliability. This invention introduces a structured replacement mechanism (such as converting user input into the form "a certain dimension has a certain value"), directly optimizing the input quality of large models, solving the problem of their lack of content modification mechanisms, and efficiently handling non-standard questions (such as colloquial or business terminology), significantly improving SQL generation accuracy while reducing the risk of error propagation. Overall, this invention achieves more efficient, lower-cost, and robust natural language question answering through a lightweight, configurable dictionary and standardized preprocessing, making it particularly suitable for data-driven interactive analysis scenarios.

[0054] This invention proposes a method for natural language question answering directly based on dimension values. Applying this method allows users to ask questions directly based on dimension values ​​during interactive question answering, automatically converting user questions into a format where a certain dimension has a certain value, which facilitates the generation of correct SQL logical expressions in large models.

[0055] like Figure 1 As shown, this invention provides a method for natural language question answering directly based on dimension values, the method comprising:

[0056] Step 101: Select the corresponding data view based on the subject domain of the user's question in natural language form;

[0057] Preferably, the data view includes: dictionary configuration and data replacement configuration;

[0058] The dictionary configuration is used to configure the dictionary items corresponding to the dimensions;

[0059] The data view is used to configure the content to be replaced by the part of speech, including: part of speech, replacement word, whether to delete special words, condition words for special word processing, and part of speech for special word processing;

[0060] Data replacement configuration is used to convert user business terms or colloquialisms into standard metrics or dimensions.

[0061] The specific content of creating a data view in this invention includes:

[0062] In semantic modeling, select a topic domain, create a data view on the data view management page, configure the Chinese name, English name and description of the data view, then select the semantic model under the current topic domain, automatically load the corresponding indicators and dimension information of the semantic model, and add the required indicators and dimensions to the data view.

[0063] The specific details of the data mapping corresponding to the configured data view include:

[0064] Configuring data mapping for the data view includes dictionary configuration, part-of-speech tagging configuration, and data replacement configuration. Dictionary configuration is used to configure the dictionary entries corresponding to each dimension. Dictionaries can be read from the data source, with a default word frequency of 100. Dictionaries can also be added from the page, and can be modified, deleted, exported, and imported. After adding a dictionary, you need to set the part-of-speech tags. When importing a dictionary, the part-of-speech tags are named by default with "n" followed by the dimension name, but you can also set the tags uniformly. After setting the part-of-speech tags, you need to click the "Apply" button to activate them. Once activated, a .txt file containing the dictionary, part-of-speech tags, and word frequency columns, and a .bin file (used for part-of-speech tagging) will be generated.

[0065] The part-of-speech tagging configuration is used to configure the content to be replaced by parts of speech. Specific configuration details include: part of speech, replacement word, special words, whether to delete special words, conditional words for handling special words, and the corresponding part of speech for handling special words. After configuring the part-of-speech tagging, it needs to be applied before it can be used. For example, if a user's question is "total output value of large-scale pig farming," the dimension is "product name," and one of the dimension's dictionaries is "large-scale pig farming," and the replacement word is configured as "product name is," the user's question will be transformed into "total output value of large-scale pig farming with product name 'product name'," thus allowing the large model to generate the correct SQL logical expression.

[0066] The data replacement configuration includes original and replacement values, used to replace user business terms or colloquialisms with standard metrics or dimensions. Before the issue is transmitted to the larger model, the corresponding data replacements will be performed based on the data replacement content. The data replacements must be effective before they can be used.

[0067] Step 102: Replace the content of the user's question with standard metrics or dimensions based on the data replacement configuration in the data view;

[0068] Preferably, the content of the user's question is replaced with standard metrics or dimensions based on the data replacement configuration in the data view, including:

[0069] For user questions in natural language format, words in the user questions are replaced based on the data replacement configuration;

[0070] The parts of speech of the replaced user questions are segmented, and the user questions are replaced with corresponding standard indicators or dimensions according to their parts of speech.

[0071] Step 103: Based on standard metrics or dimensions, convert user questions into SQL logical expressions using a large model;

[0072] Step 104: Based on the configuration information of the semantic model, translate the SQL logical expression into SQL statement;

[0073] Preferably, before translating the SQL logical expression into an SQL statement, the SQL logical expression is modified:

[0074] Check the validity of the SQL statement. If the SQL statement is found to be invalid, perform semantic correction on the SQL statement. Semantic correction includes: correction of query fields, correction of query conditions, correction of grouping conditions, and correction of query time range.

[0075] Preferably, the semantic model includes: standard metrics, dimensions, primary keys, and foreign keys;

[0076] Semantic models can be built from data sources or datasets;

[0077] When the semantic type of a semantic model is a dimension, it includes: enumeration, data time, ordinary time, primary key, and foreign key; when the semantic type of a semantic model is a metric, it includes: aggregation function.

[0078] The specific content of creating a semantic model in this invention includes:

[0079] In semantic modeling, select a topic domain and create a semantic model within that domain. Models can be created using either a data source or a dataset. When selecting a data source, first choose the database connection, then the database name and table name. Set the Chinese and English names for the semantic model. The system automatically loads all fields from the table, and you can set the semantic type for each field. For a dimension, set it to "Enumeration," "Data Time," "Normal Time," "Primary Key," or "Foreign Key." For a measure, set the aggregation function (including sum, max, min, avg, count, and count_distinct). For metrics, configure the indicator. Primary key and foreign key types are used to configure the relationships between models. When creating a model using a dataset, the corresponding field information for the dataset will be loaded, and the configuration method is the same as when creating a model using a data source.

[0080] Step 105: Based on the configuration information of the semantic model, translate the SQL logical expression into SQL statement.

[0081] This invention creates subject domains, which users can divide according to their own business scenarios;

[0082] Create a semantic model, which includes metrics, dimensions, primary keys, and foreign keys, within the subject domain.

[0083] Create a data view. Under the subject area, create a data view that includes metrics and dimensions from multiple semantic models.

[0084] Configure the data mapping corresponding to the data view, including dictionary, part-of-speech tagging, and data replacement;

[0085] In Assistant Management, create assistants and configure the large models and data views used by them;

[0086] On the Smart Question Data page, select Assistant. In the interactive question and answer window under Assistant, ask questions using natural language.

[0087] The system converts user questions into SQL logical expressions in the background using a large model, then modifies the SQL logical expressions, and finally translates them into SQL statements that can be executed on the physical data model. The system then executes the SQL queries and returns the data to the front-end page for display.

[0088] The specific steps for creating an assistant in assistant management include:

[0089] The assistant configuration page comprises four parts: basic information, large model configuration, tool configuration, and memory management. Basic information configuration includes the assistant name, whether input suggestions are enabled, whether simplified mode is enabled, whether debugging information is enabled, and sample questions. Large model configuration includes configurations for semantic SQL parsing and multi-turn dialogue rewriting; select the specific large model and then configure its prompts. The tool management page configures the tools the assistant will use; the primary tool type is Text2SQL datasets, and the data view is the previously configured data view. The memory management page includes all historical questions, pattern mappings, semantic SQL, etc. Correctly answered questions can be used as Few-shot examples to improve question-answering accuracy.

[0090] The specific content of the query data generated and displayed by the backend based on the user's question is as follows:

[0091] First, user questions in natural language format are matched against the corresponding data view in the assistant's database. Based on the metrics and dimensions in the question, the data view that best matches the current question is found. After finding the data view, the user question is replaced with data replacement data configured in the data view. This replaces user business terms or colloquialisms with standard metrics or dimensions, making it easier for the large model to generate SQL logical expressions more accurately. Then, it is determined whether to perform multi-round session rewriting. If multi-round session rewriting is performed, the questions from the previous round and the current round are sent together to the large model, which then rewrites the user question.

[0092] This invention performs pattern mapping on user questions to identify the database type corresponding to the current question, the data date field, and the semantic indicators and dimensions related to the question (including detailed semantic information corresponding to the indicators and dimensions). To address the issue that large models might not be able to identify the specific dimension corresponding to the question's dimension value, the user question is preprocessed. First, the corresponding part-of-speech replacement configuration array is retrieved based on the data view ID. Then, all indicators identified in the pattern mapping step are iterated through, and the indicator names in the user question are deleted. Next, all dimensions identified in the pattern mapping step are iterated through, and the dimension names in the user question are deleted. Then, the user question after deleting indicators and dimensions is segmented according to part-of-speech, resulting in an array of segmented objects. Each segmented object includes the segmentation, part-of-speech, and word frequency. The segmented object array is iterated through, checking whether the current part-of-speech is already included in the processed part-of-speech array. If it is, the next segmented object is processed, ensuring that each type of part-of-speech is processed only once. Finally, it is checked whether the part-of-speech corresponding to the segmented word exists in the part-of-speech configuration of the data view. If it does not exist, the next segmented object is processed. The process involves determining whether the segment corresponding to the current segmentation object is a special word. If it is, it checks if the corresponding conditional word exists in the user question. If it exists and the conditional word deletion setting is true, the current segment is deleted from the user question. Otherwise, it checks if the corresponding part-of-speech tag exists. If it exists and the conditional word deletion setting is true, the current segment is also deleted from the user question. In other cases, the segment in the user question is replaced with the replacement word in the segmentation object plus the segment of the segmentation object, i.e., in the form of a certain dimension having a certain value. Finally, the current part-of-speech tag is added to the processed part-of-speech tag array.

[0093] This invention retrieves all Few-shot examples from the cache. Each Few-shot example includes information such as user question, schema mapping, and semantic S2SQL. If it does not exist in the cache, it is retrieved from the Few-shot example configuration data. The system sets the number of Few-shot examples to N, and then finds the N examples that best match the current user question. The metrics, dimensions, and Few-shot examples included in the current question are then passed to the main model, which generates the SQL logical expression.

[0094] The SQL logic generated by the large model in this invention may contain errors, necessitating semantic correction. This involves checking the legality of the semantic query statements and correcting and optimizing any invalid information. Semantic modification includes correcting query fields, query conditions, grouping conditions, and query time ranges, resulting in higher accuracy of the corrected SQL logic.

[0095] The "large model" in this invention refers to commonly used large models. Any large model capable of generating SQL logical expressions can be used. There are no special requirements for specific large models. For example, large models like Qianwen and OpenAI can be deployed privately or through online large model interfaces. For Qianwen, a version 14b or higher is required; otherwise, accuracy will be affected.

[0096] Finally, based on the configuration information of the semantic model, the corrected SQL logical expression is translated into an SQL statement that can be executed on the physical data model. Then, the SQL statement is executed and the query results are returned to the front-end page. The front-end page selects appropriate controls for display, including indicator cards, bar charts, line charts, and tables.

[0097] The physical data model of this invention refers to the database type. When the data source is a MySQL database, it will generate executable SQL statements in MySQL syntax. When the data source is an Oracle database, it will generate executable SQL statements in Oracle syntax.

[0098] This invention configures data mapping based on data views. After a user asks a question through an interactive dialog window, the system backend finds the corresponding data view based on the indicators and dimensions in the question. Then, based on the data replacement configuration corresponding to the data view, it converts the business terms or colloquialisms in the user's question into indicators or dimensions configured in the semantic model. The data-replaced question is then sent to the large model, which can improve the accuracy of the large model in generating SQL logical expressions, thereby improving the accuracy of question and answer.

[0099] After pattern mapping, this invention removes the existing indicator and dimension names from the user question, then segments the user question after removing the indicators and dimensions according to part-of-speech. It iterates through the segmentation object array and replaces the segmented words in the user question with a certain value for a certain dimension according to the part-of-speech replacement configuration corresponding to the data view. In other words, it converts non-standard questions into standard questions. Then, it passes the standard question form to the large model, which can improve the accuracy of the large model in generating SQL logic, thereby improving the accuracy of natural language question answering.

[0100] This invention proposes a method for natural language question answering directly based on dimension values, aiming to facilitate interactive dialogue analysis using natural language and improve user efficiency and experience. This invention configures replacement words according to data views, replacing user business terms or colloquialisms with standard indicators or dimensions. By adding a dictionary corresponding to a dimension on the page or reading it from the data source, setting the part-of-speech tag in the dictionary, and then configuring the replacement words corresponding to the part-of-speech tag, a certain dimension value is replaced with a value for that dimension. When processing natural language questions, the words in the question are first replaced according to the configured replacement words; then the question is segmented according to the part-of-speech tag, and the corresponding indicator or dimension is found based on the part-of-speech tag, and replaced with a value for that dimension. This form of question is then passed to the large model, which improves the accuracy of the large model in generating SQL logical expressions, thereby improving the accuracy of natural language question answering.

[0101] The core benefit of this invention lies in significantly improving the user-friendliness and ease of use of natural language question answering. The system can accurately understand questions posed by users using business terminology or everyday colloquialisms, and automatically convert them into standardized metrics and dimensions defined in the underlying semantic model. This allows users to engage in natural language question answering as if in a casual conversation, without needing to master complex database structures or technical jargon.

[0102] This invention employs rigorous data replacement, word segmentation based on part-of-speech tags, and structured reconstruction (converting to the form of "a certain dimension is a certain value") to transform non-standard questions into standard questions. The system can accurately capture the true intent of the user's query. Combined with Few-shot examples to guide the large model to generate the correct logical expression, it significantly improves the accuracy and reliability of question-and-answer results, allowing non-technical users to efficiently and confidently obtain the data insights they need.

[0103] In terms of technical implementation, this invention creatively adopts a multi-stage processing and SQL logical intermediate layer design, which effectively overcomes the shortcomings of traditional end-to-end NL2SQL models that are prone to illusions, syntax errors or semantic deviations, and greatly ensures the accuracy, executability and correctness of the generated SQL and the final query results.

[0104] Figure 3 This is a system architecture diagram for natural language question answering based directly on dimension values ​​according to a preferred embodiment of the present invention.

[0105] like Figure 3 As shown, this invention provides a system for natural language question answering directly based on dimension values. The system includes:

[0106] Selection unit 301 is used to select the corresponding data view based on the subject domain of the user question in natural language form.

[0107] Replacement unit 302 is used to replace the content of the user question with standard indicators or dimensions based on the data replacement configuration in the data view.

[0108] Transformation unit 303 is used to convert user questions into SQL logical expressions based on standard metrics or dimensions through a large model.

[0109] Translation unit 304 is used to translate SQL logical expressions into SQL statements based on configuration information from a semantic model;

[0110] Execution unit 305 is used to execute SQL statements based on the physical data model and return the query results of the SQL statements.

[0111] Preferably, the replacement unit 302 is used to perform data replacement on the user question based on the data replacement configuration in the data view, replacing the user question with a standard indicator or dimension, and is also used to:

[0112] For user questions in natural language format, words in the user questions are replaced based on the data replacement configuration;

[0113] The parts of speech of the replaced user questions are segmented, and the user questions are replaced with corresponding standard indicators or dimensions according to their parts of speech.

[0114] Preferably, the semantic model includes: standard metrics, dimensions, primary keys, and foreign keys;

[0115] Semantic models can be built from data sources or datasets;

[0116] When the semantic type of a semantic model is a dimension, it includes: enumeration, data time, ordinary time, primary key, and foreign key; when the semantic type of a semantic model is a metric, it includes: aggregation function.

[0117] Preferably, the data view includes: dictionary configuration and data replacement configuration;

[0118] The dictionary configuration is used to configure the dictionary items corresponding to the dimensions;

[0119] The data view is used to configure the content to be replaced by the part of speech, including: part of speech, replacement word, whether to delete special words, condition words for special word processing, and part of speech for special word processing;

[0120] Data replacement configuration is used to convert user business terms or colloquialisms into standard metrics or dimensions.

[0121] Preferably, the conversion unit 303 is further configured to modify the SQL logical expression before translating it into an SQL statement:

[0122] Check the validity of the SQL statement. If the SQL statement is found to be invalid, perform semantic correction on the SQL statement. Semantic correction includes: correction of query fields, correction of query conditions, correction of grouping conditions, and correction of query time range.

[0123] The preferred embodiment of the present invention provides a system for natural language question answering directly based on dimension values, which corresponds to another preferred embodiment of the present invention providing a method for natural language question answering directly based on dimension values. These will not be described in detail here.

[0124] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0125] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0129] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0130] The invention has been described with reference to a few embodiments. However, as will be known to those skilled in the art, and as defined in the appended claims, other embodiments besides those disclosed above fall equivalently within the scope of the invention.

[0131] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the art, unless otherwise expressly defined herein. All references to “a / the / the [device, component, etc.]” ​​are openly interpreted as at least one instance of said device, component, etc., unless otherwise expressly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed unless explicitly stated otherwise.

Claims

1. A method for natural language question answering directly based on dimension values, the method comprising: Select the corresponding data view based on the subject domain of the user's question in natural language format; Based on the data replacement configuration in the data view, the content of the user question is replaced with standard indicators or dimensions. Based on the aforementioned standard metrics or dimensions, user questions are converted into SQL logical expressions using a large model. Based on the configuration information of the semantic model, the SQL logical expression is translated into an SQL statement; The SQL statement is executed based on the physical data model, and the query results of the SQL statement are returned.

2. The method according to claim 1, wherein the step of performing data replacement on the user issue based on the data replacement configuration in the data view, replacing the user issue with a standard indicator or dimension, includes: For user questions in natural language format, words in the user questions are replaced based on the data replacement configuration; The part-of-speech tag of the replaced user question is segmented, and the user question is replaced with the corresponding standard indicator or dimension according to the part-of-speech tag.

3. The method according to claim 1, wherein the semantic model comprises: Standard metrics, dimensions, primary keys, and foreign keys; The semantic model can be established using data sources or datasets; When the semantic type of the semantic model is a dimension, it includes: enumeration, data time, ordinary time, primary key, and foreign key; when the semantic type of the semantic model is a metric, it includes: aggregation function.

4. The method according to claim 1, wherein the data view comprises: Dictionary configuration and data replacement configuration; The dictionary configuration is used to configure dictionary items corresponding to dimensions; The data view is used to configure the content to be replaced by the part of speech, including: part of speech, replacement word, whether to delete special words, condition words corresponding to special word processing, and part of speech corresponding to special word processing; The data replacement configuration is used to convert user business terms or colloquialisms into standard metrics or dimensions.

5. The method according to claim 1, further comprising modifying the SQL logical expression before translating it into an SQL statement: Check if the SQL statement is valid; When the SQL statement is found to be invalid, semantic correction is performed on the SQL statement; the semantic correction includes: Query field correction, query condition correction, grouping condition correction, and query time range correction.

6. A system for natural language question answering directly based on dimension values, the system comprising: The selection unit is used to select the corresponding data view based on the subject domain of the user's question in natural language form. The replacement unit is used to perform data replacement on the content of the user question based on the data replacement configuration in the data view, replacing the content of the user question with standard indicators or dimensions. The conversion unit is used to convert user questions into SQL logical expressions using a large model based on the standard indicators or dimensions. The translation unit is used to translate the SQL logical expression into an SQL statement based on the configuration information of the semantic model; The execution unit is used to execute the SQL statement based on the physical data model and return the query results of the SQL statement.

7. The system according to claim 6, wherein the replacement unit is configured to perform data replacement on the user question based on the data replacement configuration in the data view, replacing the user question with a standard indicator or dimension, and is further configured to: For user questions in natural language format, words in the user questions are replaced based on the data replacement configuration; The part-of-speech tag of the replaced user question is segmented, and the user question is replaced with the corresponding standard indicator or dimension according to the part-of-speech tag.

8. The system according to claim 6, wherein the semantic model comprises: Standard metrics, dimensions, primary keys, and foreign keys; The semantic model can be established using data sources or datasets; When the semantic type of the semantic model is a dimension, it includes: enumeration, data time, ordinary time, primary key, and foreign key; when the semantic type of the semantic model is a metric, it includes: aggregation function.

9. The system according to claim 6, wherein the data view comprises: Dictionary configuration and data replacement configuration; The dictionary configuration is used to configure dictionary items corresponding to dimensions; The data view is used to configure the content to be replaced by the part of speech, including: part of speech, replacement word, whether to delete special words, condition words corresponding to special word processing, and part of speech corresponding to special word processing; The data replacement configuration is used to convert user business terms or colloquialisms into standard metrics or dimensions.

10. The system according to claim 6, wherein the conversion unit is further configured to modify the SQL logical expression before translating the SQL logical expression into an SQL statement: Check the validity of the SQL logical expression. If the SQL logical expression is found to be invalid, perform semantic correction on the SQL logical expression. The semantic correction includes: Query field correction, query condition correction, grouping condition correction, and query time range correction.

Images