Intelligent question answering method, device and system based on semantic data model and large model

Abstract

The application discloses an intelligent question answering method, device and system based on a semantic data model and a large model, and belongs to the field of artificial intelligence. After receiving a current question of a user, a task plan corresponding to the current question of the user is obtained based on a large model, then MQL information extraction is performed based on the task plan to obtain a target MQL, then the target MQL is converted into an SQL query statement by a word semantic data model, query is performed based on the SQL query statement to obtain reply data, and finally the reply data is returned to the user. The technical scheme of the application converts natural language into MQL by using a large model, and then converts the MQL into SQL by using a semantic data model. Compared with a traditional scheme of directly converting natural language into SQL, the technical scheme does not need to train data, and the large model can accurately identify the intention of the user, so that the MQL obtained has a unified semantic layer and provides a standard caliber, and the accuracy of reply is greatly improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to an intelligent questioning method, apparatus and system based on semantic data models and large models. Background Technology

[0002] In today's era of rapid internet development, big data analytics has become a trend, adopted by various industries to extract valuable information and assist businesses in making business decisions. However, in daily work, big data analytics faces two major challenges: first, it has a high barrier to entry, requiring both professional development and data analysis skills; second, it suffers from low timeliness and high personalization costs, necessitating professional analysis to generate specific reports.

[0003] Therefore, with the development of artificial intelligence technology, the research and development direction of semantic analysis based on large models and retrieval technology and transformed into SQL and chart forms for data queries has received attention and development.

[0004] Existing intelligent query methods all generate query SQL directly based on Text2SQL technology. However, this approach requires a large amount of labeled data for training, and for cases involving new table logic, the data needs to be re-labeled and the model retrained. Furthermore, the resulting model has many defects when used. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this application provides an intelligent query method, device, and system based on semantic data models and large models. This addresses the problem that existing intelligent query methods directly generate query SQL based on Text2SQL technology, but this approach requires a large amount of labeled data for training. Furthermore, for cases involving new table logic, data needs to be re-labeled and the model needs to be retrained, and the resulting model has many defects when used.

[0006] The technical solution adopted by this application to solve its technical problem is:

[0007] Firstly, a smart data retrieval method based on semantic data models and large models is provided, including:

[0008] Receive the user's current question;

[0009] Based on the large model, the task plan corresponding to the user's current question is obtained;

[0010] Based on the task planning, MQL information is extracted to obtain the target MQL;

[0011] The target MQL is transformed into an SQL query statement using a semantic data model;

[0012] The SQL query statement is used to retrieve the response data, which is then returned to the user.

[0013] As an optional implementation of this application, the step of obtaining the task planning corresponding to the user's current question based on a large model includes:

[0014] The user's current question is processed so that the large model can identify the user's intent;

[0015] The task type is determined based on the user's intent;

[0016] Based on the task type, a task plan is obtained. The task plan is a structured task list, and each task in the task list specifies the specific processing goal and scope.

[0017] As an optional implementation of this application, the processing of the user's current question includes:

[0018] Based on the user's business knowledge base and pattern matching library, the natural language expression of the user's current question is replaced with a standardized equivalent question.

[0019] As an optional implementation of this application, the processing of the user's current question includes:

[0020] Determine whether there is a time element in the user's current question;

[0021] If it does not exist, then obtain the candidate preceding question containing time elements and its corresponding receiving time;

[0022] The time value of the candidate preceding question that was received latest is added to the user's current question.

[0023] As an optional implementation of this application, the processing of the user's current question includes:

[0024] Obtain the dimension values ​​from the user's current question;

[0025] If a target dimension value corresponds to at least two dimensions, then the target dimension value is retrieved and assembled into a complete sentence list, which is then returned to the user for confirmation.

[0026] As an optional implementation of this application, the step of extracting MQL information based on the task planning to obtain the target MQL includes:

[0027] Metadata retrieval information is obtained by performing metadata retrieval on each task in the task plan;

[0028] Provide the prompt words, MQL field descriptions and definitions, metadata retrieval information and samples to the large model to obtain the MQL information for the current round with a preset structure;

[0029] The target MQL is obtained based on the MQL information of the current round.

[0030] As an optional implementation of this application, obtaining the target MQL based on the MQL information of the current round includes:

[0031] Determine whether the MQL information of the current round includes both time and indicator elements;

[0032] If both time and indicator elements are included, the target MQL is obtained based on the MQL information of the current round; if neither time nor indicator elements are included, the corresponding time or indicator elements in the historical MQL are used as the time or indicator elements in the MQL information of the current round, and the dimensions included in the current round are compared with the dimensions in the historical MQL.

[0033] If the same dimension exists, replace the historical dimension value with the current dimension value; if it is a new dimension, append the current dimension value as a filter condition to the historical MQL to obtain the target MQL.

[0034] As an optional implementation of this application, it also includes:

[0035] After each task is completed, the result status information of the task is obtained. The result status information includes: the completion progress of the task, the details of the processing result, and whether any abnormalities have occurred.

[0036] Based on the result status information, determine whether a new task needs to be added;

[0037] If not needed, the task execution results will be integrated to obtain the response data and returned to the user; if needed, a new supplementary task will be generated and added to the task list.

[0038] Secondly, a smart data query device based on semantic data models and large models is provided, including:

[0039] The user question receiving module is used to receive the user's current question.

[0040] The task planning acquisition module is used to obtain the task plan corresponding to the user's current question based on the large model.

[0041] The MQL information extraction module is used to extract MQL information based on the task planning to obtain the target MQL;

[0042] The SQL statement conversion module is used to convert the target MQL into an SQL query statement using a semantic data model.

[0043] The SQL query execution module is used to perform a query based on the SQL query statement to obtain response data and return the response data to the user.

[0044] Thirdly, a smart data query system based on semantic data models and large models is provided, including:

[0045] At least one processor and at least one memory;

[0046] The memory stores the executable instructions of the processor;

[0047] The processor is configured to execute the intelligent questioning method based on semantic data models and large models as described above.

[0048] Beneficial effects:

[0049] This application provides an intelligent question-and-answer method, apparatus, and system based on a semantic data model and a large-scale model. After receiving a user's current question, a task plan corresponding to the user's current question is obtained based on a large-scale model. Then, MQL information is extracted based on the task plan to obtain the target MQL. Subsequently, a word semantic data model converts the target MQL into an SQL query statement. The SQL query statement is then used to obtain response data, which is finally returned to the user. This application's technical solution uses a large-scale model to convert natural language into MQL, and then uses a semantic data model to convert the MQL into SQL. Compared with traditional methods that directly convert natural language into SQL, this eliminates the need for data training. Furthermore, the large-scale model can accurately identify user intent, and the resulting MQL, with its unified semantic layer, provides a standardized definition, significantly improving response accuracy. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart of an intelligent question-and-answer method based on a semantic data model and a large model provided in an embodiment of this application;

[0052] Figure 2 This is a schematic diagram of an intelligent question-and-answer device based on a semantic data model and a large model, provided in an embodiment of this application.

[0053] Figure 3 This is a schematic diagram illustrating the implementation process of an intelligent data query system based on a semantic data model and a large model, provided in an embodiment of this application.

[0054] Figure 4 This is a flowchart of an MQL slot update provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] Existing intelligent query solutions have obvious, unsolvable flaws:

[0057] 1. The ambiguity and diversity of natural language, the lack of a standard definition, and the inability to correctly understand user intent.

[0058] 2. Optimal SQL generation performance cannot be guaranteed, especially when querying large-scale databases, where performance may become a bottleneck, requiring optimization of query generation and execution efficiency.

[0059] 3. Performance degrades significantly in multi-round scenarios, making it impossible to continuously track user intent.

[0060] 4. The generated SQL cannot guarantee accuracy and performance requirements, and cannot be directly used by business users.

[0061] To solve the above problems, refer to Figure 1 This application provides an intelligent question-and-answer method based on semantic data models and large models, including:

[0062] S11: Receive the user's current question. The user's current question is a question input using natural language. It should be noted that this application pre-stores user-defined metrics and dimensions.

[0063] S12: Based on the large model, obtain the task plan corresponding to the user's current question; specifically as follows:

[0064] The user's current question is processed so that the large model can identify the user's intent;

[0065] The task type is determined based on the user intent; the task type is predefined based on business requirements and can be specifically set according to business needs. Examples include: Metric-based task: Intending to ask questions about metrics and dimensions within a specified time range, such as "Today's sales revenue," or "What percentage of sales revenue did the Nanshan store generate yesterday?" Attribution analysis task: Tasks that perform attribution analysis on metrics and dimensions, such as "Why did sales revenue decline this quarter?"

[0066] Based on the task types, a task plan is derived. This task plan is a structured list of tasks, where each task specifies its concrete processing objective and scope. The general divisions for each task type are similar, and the task plan can be obtained by combining the specific data mentioned in the question.

[0067] It's understandable that user-inputted questions may use different words to express the same meaning due to personal habits, etc. Although the large model can recognize these words, they may differ from the data stored in the database. Therefore, to facilitate large model recognition and ensure the accuracy of subsequent results, it is necessary to process the user's current question.

[0068] For example, the natural language expression of the user's current question can be replaced with a standardized equivalent question based on the user's business knowledge base and pattern matching library.

[0069] Alternatively, determine if a time element exists in the user's current question; if not, obtain candidate preceding questions containing time elements and their corresponding receiving times; add the time value of the candidate preceding question with the latest receiving time to the user's current question. Because in intelligent question counting, users engage in multiple rounds of questioning and answering. For example, a user might initially ask: "What was the sales volume of men's shoes in June of this year?" After answering this question, the user might simply input "What about women's shoes?", implicitly referring to the sales volume of women's shoes in June of this year. If subsequent processing only uses "What about women's shoes?", the resulting result is not what the user actually wants. Therefore, this application rewrites the current user query time context to make the question more consistent with the user's true current intent.

[0070] Alternatively, the dimension values ​​in the user's current query can be retrieved. If a target dimension value corresponds to at least two dimensions, the target dimension value is retrieved and assembled into a complete sentence list, which is then returned to the user for confirmation. The existence of a target dimension value corresponding to at least two dimensions indicates that the target dimension value has multiple mappings, thus representing multiple user intentions. For example, the query "Sales volume of B" has two dimensions: "Store B" and "Salesperson named B" (both are preset dimensions in this application). Therefore, a query with both dimensions can be returned to the user for confirmation.

[0071] S13: Based on the task plan, extract MQL information to obtain the target MQL; specifically, perform metadata retrieval for each task in the task plan to obtain metadata retrieval information; for example, use full matching and substring matching algorithms to find the basic indicators involved in the current problem, such as: sales revenue and sales volume in the same period yesterday, and find the system-configured sales revenue and sales volume indicators in the same period. Then find the dimensions and dimension value examples corresponding to the indicator configuration.

[0072] It's important to note that MQL (Metrics Query Language) is a specialized language for querying, retrieving, analyzing, and aggregating monitoring metric data. It features structured semantics, facilitating validation by converting natural language data into MQL. Furthermore, MQL categorizes data into time elements, dimensional elements, and metric elements. The calculation method for each metric is predefined, making it suitable for various business scenarios.

[0073] The system provides prompts, MQL field descriptions, metadata retrieval information, and examples to the large model to obtain the MQL information for the current round with a pre-defined structure. In other words, it defines a unified MQL query structure, then provides the definition of each field in the MQL, along with prompts, source data retrieval information, and examples, to the large model, allowing the large model to obtain the MQL information for the current round. For example, the unified MQL query structure includes: a list of query metrics, analysis dimension configuration, result sorting configuration, result set quantity limits, dimension filtering configuration, and date dimension-specific configuration.

[0074] The prompt words are different predefined prompt words based on the specific task type.

[0075] The MQL field description definition includes the role of each field in the MQL query structure and the extraction / configuration rules. Taking the existing MQL query structure as an example, the purpose of the query indicator list is to define the core business indicators (such as sales revenue and order volume) that need to be calculated and analyzed, which are the query targets of MQL. The extraction rules are in list format, and the element is a "unified semantic indicator name" (which needs to be predefined in the underlying metadata, such as "sales revenue" corresponding to the SUM(sale_amount) calculation logic of the underlying data source). The purpose of the analysis dimension configuration is to define "from which angle to split the data". The configuration rules are in list format, and the element is a "unified semantic dimension name" (such as "city", "month", "product category"). It supports multi-dimensional combinations to achieve multi-level analysis. For example, the dimension ["province", "city", "store"] needs to be associated with data types (defined in the semantic dictionary, such as "city" as string type and "month" as date type). The purpose of the result sorting configuration is to define the sorting rules of the query results and control the order of data display. The configuration rules are in array format, with each element representing a single sorting condition, supporting multiple sorting conditions. A single condition includes: `col`: the sorting field (must be a field from metrics or dimensions, such as "sales revenue" or "city"); `orderBy`: the sorting direction (ASC ascending, DESC descending). The result set quantity limit controls the number of returned results, preventing excessive data volume from causing query performance degradation or front-end rendering lag. The configuration rules are numeric, with default values ​​configurable in the system (e.g., default 100); value range: 0 < limit ≤ 1000 (exceeding the limit requires special application to prevent large data volume queries from overwhelming the system); commonly used in "Top N" scenarios (e.g., "Retrieve the top 10 cities by sales revenue"). The dimension filtering configuration filters data based on the specific values ​​of the dimension fields specified in the analysis dimension configuration. The configuration rules are as follows: list format, each element is a single filtering condition, supporting multiple condition combinations; a single condition includes: col: the filtering dimension (must be a field in the analysis dimension configuration, such as "city" or "product category"); operator: the operator (common: IN (includes), NOT IN (does not include), EQ (equals), NEQ (not equals), LIKE (fuzzy matching), etc.); value: the filtering value (must match the dimension data type, such as ["Beijing", "Shanghai"] for string types, and [100, 200] for numeric types). The date dimension-specific configuration allows for separate configuration of date-related filtering and granularity. Since date is the most commonly used analysis dimension, it requires special handling to support flexible time range queries.Configuration rules: Includes two subfields: granularity: date granularity (time splitting unit), optional values: ALL (no splitting, take full data), YEAR (year), QUARTER (quarter), MONTH (month), WEEK (week), DAY (day), HOUR (hour); range: date range (filter time interval), such as ["2024-06-25 00:00:00", "2024-07-01 23:59:59"].

[0076] The examples are pre-set MQL extraction examples for each scenario.

[0077] The target MQL is obtained based on the MQL information of the current round.

[0078] As a preferred implementation of this application, obtaining the target MQL based on the MQL information of the current round includes:

[0079] Determine whether the MQL information of the current round includes both time and indicator elements; that is, the elements in the MQL information in this application embodiment are divided into three types: time, indicator (the parameter used for the final calculation), and dimension (the parameter used to describe the indicator). Of course, time can also be used as a dimension, but since time is the most commonly used analysis dimension in intelligent data analysis, it is set separately.

[0080] If both time and metric elements are included, the target MQL is obtained based on the MQL information of the current round. The inclusion of both time and metric elements indicates that the current MQL structure is complete and can be directly returned. Conversely, if both time and metric elements are not included—for example, missing the metric element means the final target query is missing and cannot be returned, while missing the time element may result in too much MQL information being returned.

[0081] Therefore, if time and metric elements are not included simultaneously, the corresponding time or metric elements from the historical MQL are used as the time or metric elements in the current round's MQL information. The dimensions included in the current round are compared with the dimensions in the historical MQL. This is because if time and metric elements are not included simultaneously, it means that the previous round's question already included the missing time or metric elements, so the corresponding elements from the historical MQL can be directly used as the elements in the current round. It can be understood that the historical MQL refers to the MQL closest to the current round that contains the corresponding missing elements (the time and / or metric elements missing in this round). The comparison of dimensions is because new dimensions may be added. For example, a user's historical question might be "What was the sales volume (metric) in Shenzhen (dimension) in June (time element)?", while the current question is "What about May?". In this case, the metric is missing, but the dimension is not new. However, if the current question is "What about Guangzhou?", both time and metric are missing, but the dimension is new. Therefore, it is necessary to determine whether the dimensions are the same.

[0082] If the same dimension exists, replace the historical dimension value with the current dimension value; if it is a new dimension, append the current dimension value as a filter condition to the historical MQL to obtain the target MQL.

[0083] In addition, after each task is completed, the result status information of the task is obtained, including: the completion progress of the task, the details of the processing result, and whether any abnormality occurred.

[0084] Based on the result status information, determine whether a new task needs to be added;

[0085] If not needed, the task execution results will be integrated to obtain the response data and returned to the user; if needed, a new supplementary task will be generated and added to the task list.

[0086] S14: The target MQL is converted into an SQL query statement using a semantic data model; the semantic data model is pre-trained, but since MQL is structured and unified semantic data, it is easier to train than natural semantic data.

[0087] S15: Perform a query based on the SQL query statement to obtain the response data, and return the response data to the user.

[0088] It should be noted that any process or method description in the flowchart or otherwise described herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which the embodiments of this application pertain.

[0089] Furthermore, in the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0090] The intelligent question-and-answer method based on semantic data models and large models provided in this application, after receiving the user's current question, obtains the task plan corresponding to the user's current question based on the large model, then extracts MQL information based on the task plan to obtain the target MQL, and then the semantic data model converts the target MQL into an SQL query statement, performs a query based on the SQL query statement to obtain the response data, and finally returns the response data to the user. This technical solution uses a large model to convert natural language into MQL, and then uses a semantic data model to convert the MQL into SQL. Compared with the traditional solution of directly converting natural language into SQL, it does not require training on the data; moreover, the large model can accurately identify the user's intent, and the resulting MQL with a unified semantic layer provides a standard, greatly improving the accuracy of the response.

[0091] Based on the same inventive concept, such as Figure 2 As shown, this application provides an intelligent question-and-answer device 20 based on a semantic data model and a large model, comprising:

[0092] User question receiving module 21 is used to receive the user's current question.

[0093] Task planning acquisition module 22 is used to obtain the task plan corresponding to the user's current question based on the large model;

[0094] The task planning based on the large model to obtain the user's current question includes:

[0095] The user's current question is processed so that the large model can identify the user's intent; the task type is determined based on the user's intent; a task plan is obtained based on the task type, and the task plan is a structured task list, in which each task in the task list specifies the specific processing goal and scope.

[0096] The processing of the user's current question includes:

[0097] Based on the user's business knowledge base and pattern matching library, the natural language expression of the user's current question is replaced with a standardized equivalent question.

[0098] And / or, determine whether there is a time element in the user's current question; if not, obtain the candidate preceding question containing the time element and its corresponding reception time; add the time value of the candidate preceding question with the latest reception time to the user's current question.

[0099] And / or, obtain the dimension values ​​in the user's current question; if there are at least two dimensions corresponding to the target dimension value, retrieve the target dimension value and assemble it into a complete sentence list and return it to the user for confirmation.

[0100] Also includes:

[0101] After each task is completed, the result status information of the task is obtained. The result status information includes: the completion progress of the task, the details of the processing result, and whether any abnormalities have occurred.

[0102] Based on the result status information, determine whether a new task needs to be added;

[0103] If not needed, the task execution results will be integrated to obtain the response data and returned to the user; if needed, a new supplementary task will be generated and added to the task list.

[0104] MQL information extraction module 23 is used to extract MQL information based on the task planning to obtain the target MQL;

[0105] The step of extracting MQL information based on the task planning to obtain the target MQL includes:

[0106] Metadata retrieval information is obtained by performing metadata retrieval on each task in the task planning; prompt words, MQL field descriptions and definitions, metadata retrieval information and samples are provided to the large model to obtain the MQL information of the current round with a preset structure; the target MQL is obtained based on the MQL information of the current round.

[0107] Specifically, determine whether the MQL information of the current round includes both time elements and indicator elements;

[0108] If both time and indicator elements are included, the target MQL is obtained based on the MQL information of the current round; if neither time nor indicator elements are included, the corresponding time or indicator elements in the historical MQL are used as the time or indicator elements in the MQL information of the current round, and the dimensions included in the current round are compared with the dimensions in the historical MQL; if the same dimension exists, the historical dimension value is replaced with the current dimension value; if it is a new dimension, the current dimension value filtering condition is appended to the historical MQL to obtain the target MQL.

[0109] SQL statement conversion module 24 is used to convert the target MQL into an SQL query statement using a semantic data model.

[0110] The SQL query execution module 25 is used to perform a query based on the SQL query statement to obtain response data and return the response data to the user.

[0111] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the intelligent query method based on semantic data models and large models provided in any of the above embodiments.

[0112] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0113] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0114] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0115] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0116] The computer-readable storage medium provided in this application embodiment stores a computer program. When executed by a processor, the computer program implements the steps of the intelligent question-and-answer method based on a semantic data model and a large model as provided in any of the above embodiments. Thus, after receiving the user's current question, a task plan corresponding to the user's current question is obtained based on the large model. Then, based on the task plan, MQL information is extracted to obtain the target MQL. Subsequently, the word semantic data model converts the target MQL into an SQL query statement. Based on the SQL query statement, a query is performed to obtain response data, and finally, the response data is returned to the user. The technical solution of this application uses a large model to convert natural language into MQL, and then uses a semantic data model to convert the MQL into SQL. Compared with the traditional solution of directly converting natural language into SQL, no training on the data is required. Furthermore, the large model can accurately identify user intent, and the resulting MQL with a unified semantic layer provides a standard, greatly improving the accuracy of the response.

[0117] Based on the same inventive concept, this application also provides an intelligent question-and-answer system based on semantic data models and large models, including:

[0118] At least one processor and at least one memory;

[0119] The memory stores the executable instructions of the processor;

[0120] The processor is configured to execute the intelligent query method based on semantic data models and large models provided in the above embodiments.

[0121] A schematic diagram of the overall technical solution of the system, such as Figure 3 As shown, it mainly consists of three modules: the intent recognition layer, the MQL information extraction layer, and the data query layer.

[0122] I. Intent Recognition Layer

[0123] The consciousness recognition layer mainly includes modules such as intent enhancement, time rewriting, retrieval and recall, and task planning.

[0124] Intent Enhancement Module: Based on the user's business knowledge base and pattern matching library, the natural language expression of the input question is replaced with a standardized equivalent question, unifying the expression of the demand intent and reducing the impact of expression differences on subsequent recognition.

[0125] The time rewriting module rewrites the current user's query time based on the context, making it more consistent with the user's true intent. The reasons for including this module are twofold: 1. Different companies have different requirements for time ranges, such as whether this week is Monday to Sunday or Monday to Friday, so a specific module is needed to handle time uniformly. 2. It adheres to best practices for model understanding, especially since mixing multi-turn time recognition with other content can easily lead to errors. Unlike traditional question-answering systems, intelligent question counting typically involves multi-turn questions and answers. In this case, after a user's input in the previous round included time, metrics, and dimensions, they might simply change the metrics or dimensions to omit the time in the next round for ease of use. If the question doesn't include time, the amount of data to retrieve increases, and the final results may be inaccurate. Therefore, the time rewriting module is introduced.

[0126] The time rewriting module and the intent addition module can convert user-input natural language questions into standard structured questions.

[0127] Retrieval and Recall Module: Utilizes a built-in matching algorithm to recall dimension values ​​involved in the user's question, enabling the user to clarify their intent.

[0128] For example, using Elasticsearch's built-in matching algorithm, the steps are as follows:

[0129] 1. During initialization, store the dimension values ​​of potentially conflicting dimensions into Elasticsearch. For example, if the query asks for the sales revenue of brand A in Shenzhen today, Shenzhen can be both a region dimension and a city dimension in the user definition, so the user's intent is unclear.

[0130] 2. Elasticsearch retrieves conflicting dimension values ​​and assembles them into a complete sentence list for user clarification, such as "Today's sales figures for brand A in Shenzhen region, today's sales figures for brand A in Shenzhen city".

[0131] 3. The user confirms their selection. If they select "Today's sales of Brand A in Shenzhen", then the confirmed dimension intent is: city Shenzhen.

[0132] 4. Tailor the question: Continue to provide downstream users with questions whose intent is not yet confirmed, such as "Today's sales volume of brand A". That is, in this case, both intents are treated as user intents for subsequent actions.

[0133] Task planning module: Combining large-scale task planning, ReAct and function calling techniques, it identifies the user's true intent, selects appropriate tools, and extracts corresponding parameters for subsequent use;

[0134] 1. Initialize the following intent tasks (i.e., identify task types)

[0135] a. Indicator-based questioning: Questioning intent for indicators and dimensions within a specified time range, such as "today's sales" or "yesterday's sales percentage of Nanshan store".

[0136] b. Attribution analysis: Conduct attribution analysis on indicators and dimensions to identify the causes of fluctuations, trends, and influencing factors.

[0137] c. System Asset Information Inquiry: Inquire about asset information such as system configuration indicators and dimensions.

[0138] d. Indicator Recommendation: If there is no specific indicator for scenario a, the most relevant indicator will be recommended.

[0139] d. Refusal to answer: All other questions unrelated to the target number.

[0140] 2. The Plan-and-Execute ReAct technology has the advantage of being able to break down complex tasks by optimizing planning and task execution processes.

[0141] a. The Planning phase first receives the user's query requirements or instructions, and at the same time obtains the unified semantic metadata information that has been organized in the system. This metadata specifically includes various indicators (such as sales revenue, sales volume, etc.) and dimensional data (such as city dimension, product dimension, etc.) predefined by the user. This information will serve as the basic data support for subsequent task processing, ensuring that the understanding and processing of user needs remain consistent throughout the entire process.

[0142] b. After obtaining user input and metadata, the system will break down complex needs into a series of specific executable tasks based on the analysis of the user's true intentions (including the user's explicit needs and potential implicit needs), forming a structured task list, with each task having a clear processing goal and scope.

[0143] c. For different tasks in the task list, the system will distribute them to the corresponding specialized processing logic modules in the MQL (Metadata Query Language) information extraction layer according to the nature of the task (such as data query, indicator calculation, dimensional analysis, etc.). Different processing logic modules are optimized for specific types of tasks to ensure the accuracy and efficiency of task processing.

[0144] d. After each task is processed in the MQL information extraction layer, the result status information of the task will be updated in real time through the Single-task agent. This status information includes, but is not limited to, the task completion progress, processing result details, and whether any anomalies have occurred. The updated status information will be synchronized to the system's global status management module to provide the latest and most accurate decision-making basis for the task replanning in the next stage (Replan stage).

[0145] e. The Replan phase primarily determines subsequent actions based on the real-time status of task processing, and specifically falls into two categories:

[0146] The first scenario: If the system determines, based on the completion status and results of the current task list, that there is no need to add any new tasks (i.e., the processing results of the existing tasks have fully covered and met the user's needs), then the integrated final result will be directly fed back to the user.

[0147] The second scenario: If, after analyzing the execution status and returned results of the task list, it is found that the processing results of the existing tasks do not fully meet the user's needs (e.g., there is missing information, incomplete data, or the need for further in-depth analysis), the system will automatically generate new supplementary tasks and add them to the task list. Subsequently, the entire process will re-enter the cycle of task distribution, processing, and status updates until the processing results of all tasks fully meet the user's needs.

[0148] II. MQL Message Extraction Layer

[0149] It mainly includes the following modules:

[0150] Metadata retrieval:

[0151] 1. Use full matching and substring matching algorithms to find the basic indicators involved in the current problem, such as the sales revenue and sales volume of yesterday in the same period, and find the sales revenue and sales volume indicators configured in the system for the same period.

[0152] 2. Locate the corresponding dimensions and dimension value examples for the indicator configuration.

[0153] Dynamic Prompt and MQL Extraction:

[0154] The following are examples of unified semantic MQL query structure design and different field descriptions:

[0155]

[0156] The definitions and functions of each field are explained below:

[0157] metrics: List of metrics to query

[0158] a. Purpose: Define the core business metrics (such as sales revenue and order volume) that need to be calculated and analyzed; these are the query targets of MQL.

[0159] b. Extraction rules: List format, with elements being "unified semantic indicator names" (which need to be predefined in the underlying metadata, such as "sales amount" corresponding to the SUM(sale_amount) calculation logic of the underlying data source); supports parallel queries of multiple indicators, such as ["sales amount", "order volume", "average order value"];

[0160] dimensions: Analysis dimension configuration

[0161] a. Function: Defines "from which angle to split the data", similar to SQL's GROUP BY, such as "split sales by city" or "split order volume by month".

[0162] b. Configuration rules: List format, with elements being "unified semantic dimension names" (such as "city", "month", "product category"); supports multi-dimensional combinations to achieve multi-level analysis, such as ["province", "city", "store"] dimensions need to be associated with data types (defined in the semantic dictionary, such as "city" as a string and "month" as a date).

[0163] order: Result sorting configuration

[0164] a. Function: Defines the sorting rules for query results and controls the order in which data is displayed.

[0165] b. Configuration rules: Array format, each element is a single sorting condition, supporting multiple sorting conditions; a single condition includes: col: sorting field (must be a field in metrics or dimensions, such as "sales" or "city"); orderBy: sorting direction (ASC ascending, DESC descending).

[0166] limit: Limit on the number of result sets

[0167] a. Function: To control the number of results returned, avoiding performance degradation or front-end rendering lag caused by excessive data volume.

[0168] b. Configuration rules: Numeric type, default value can be configured in the system (e.g., default 100); value range: 0 < limit ≤ 1000 (exceeding the limit requires special application to prevent large data volume queries from overwhelming the system); commonly used in "getting Top N" scenarios (e.g., "getting the top 10 cities in sales").

[0169] filters: Dimensional filtering configuration

[0170] a. Function: Filters data based on the specific values ​​of the dimensions field, similar to the WHERE clause in SQL (filtering dimension conditions).

[0171] b. Configuration Rules: List format, each element is a single filter condition, supporting multiple condition combinations; a single condition includes: col: filter dimension (must be a field in dimensions, such as "city" or "product category"); operator: operator (common: IN (includes), NOT IN (does not include), EQ (equals), NEQ (not equals), LIKE (fuzzy matching), etc.); value: filter value (must match the dimension data type, such as ["Beijing", "Shanghai"] for string type, and [100, 200] for numeric type).

[0172] date: Dedicated configuration for the date dimension

[0173] a. Function: Configure date-related filters and granularity separately, as date is the most commonly used analysis dimension and requires special processing to support flexible time range queries.

[0174] b. Configuration Rules: Includes two subfields: granularity: date granularity (time splitting unit), optional values: ALL (no splitting, full value), YEAR, QUARTER, MONTH, WEEK, DAY, HOUR; range: date range (filter time intervals), such as ["2024-06-25 00:00:00", "2024-07-01 23:59:59"].

[0175] 2. Dynamically combine system prompts, MQL field descriptions and definitions, metadata retrieval information, and few show examples for MQL information extraction, as detailed below:

[0176] System prompt: "## Task"

[0177] You are currently an expert in extracting original information. Based on the given candidate metrics and dimensions, extract information from the input question according to the specified format.

[0178] MQL Field Description Definition: The above field definition information

[0179] Metadata retrieval information: Retrieves candidate metrics, dimensions, and dimension value information.

[0180] Few show: Pre-designed MQL extraction examples for various scenarios.

[0181] 3. MQL slot update strategy

[0182] In multi-turn dialogue scenarios, combined with user questioning habits, slot update technology in task-oriented dialogues is adopted to effectively solve the problem that excessive historical context leads to a significant decrease in intent and extraction performance.

[0183] Intelligent question-and-answer scenarios differ from general question-and-answer scenarios in that their context inheritance primarily revolves around three core elements: time, dimensions, and metrics. Therefore, when handling multi-turn interaction scenarios, it's unnecessary to rely on the complete historical context; instead, it's sufficient to update the abstracted Multi-Turn Query Language (MQL) based on the elements extracted in the current turn. This approach effectively avoids the problem of reduced element extraction accuracy due to overload of historical context information. The specific MQL update strategy is as follows: Figure 4 As shown:

[0184] 1. First, determine whether the elements extracted in the current round simultaneously contain the "time + indicator" combination. If this condition is met, then the MQL information extracted in the current round is directly used as the final effective MQL information.

[0185] 2. If the elements extracted in the current round do not meet the "time + indicator" combination, then iterate through the currently extracted MQL information. For elements that are "non-dimensional value filtered" (such as indicators, time, etc.), directly replace the corresponding content in the historical MQL with the currently extracted information.

[0186] 3. If the current round extracts a "dimensional value filter condition", then compare it with the dimensions in the historical MQL. If the same dimension exists, replace the historical dimension value with the current dimension value; if it is a new dimension, then append the current dimension value filter condition to the historical MQL.

[0187] III. Data Query Layer

[0188] The core capability is to transform the extracted MQL into the final high-performance query SQL through the SDM model, ensuring data accuracy and timeliness.

[0189] MQL to SQL conversion

[0190] A semantic data model (SDM) is a conceptual model used to represent data and its relationships. It makes full use of semantic information (i.e., the meaning of the data) to help better organize and understand the data. The core concepts of a semantic data model include entities, attributes, and relationships.

[0191] A model describes the entities and dimensions in an SDM (System Data Model), presented as a logical view mapped to entity tables, SQL, or tabular files. It primarily includes data columns, entity dimensions, and the definitions of relationships between entities. Through it, users can define complex data models, including joining multiple tables, filtering data, and adding calculated fields, providing powerful support for data analysis.

[0192] Metrics describe the metrics in SDM, including (calculated) aggregations based on model data columns or the calculation results of multiple metrics, such as sales, profits, or number of visitors. Through it, users can quickly build an enterprise-level metric system.

[0193] Compared to traditional natural language to SQL conversion, this application's solution introduces MQL, which can more accurately understand user intent through a large model and then convert it into MQL with a simpler structure. Although MQL is relatively simple in form compared to SQL, it is still a standard structured data. Therefore, using a semantic data model to convert MQL into SQL is more accurate and easier to implement than directly converting natural language to SQL.

[0194] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0195] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0196] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An intelligent question-and-answer method based on semantic data models and large models, characterized in that, include: Receive the user's current question; Based on the large model, the task plan corresponding to the user's current question is obtained; Based on the task planning, MQL information is extracted to obtain the target MQL; The target MQL is transformed into an SQL query statement using a semantic data model; The SQL query statement is used to obtain the response data, and the response data is returned to the user. The step of extracting MQL information based on the task planning to obtain the target MQL includes: Metadata retrieval information is obtained by performing metadata retrieval on each task in the task plan; Provide the prompt words, MQL field descriptions and definitions, metadata retrieval information and samples to the large model to obtain the MQL information for the current round with a preset structure; Determine whether the MQL information of the current round includes both time and indicator elements; If both time and indicator elements are included, the target MQL is obtained based on the MQL information of the current round; if neither time nor indicator elements are included, the corresponding time or indicator elements in the historical MQL are used as the time or indicator elements in the MQL information of the current round, and the dimensions included in the current round are compared with the dimensions in the historical MQL. If the same dimension exists, replace the historical dimension value with the current dimension value; if it is a new dimension, append the current dimension value as a filter condition to the historical MQL to obtain the target MQL.

2. The method according to claim 1, characterized in that: The task planning based on the large model to obtain the user's current question includes: The user's current question is processed so that the large model can identify the user's intent; The task type is determined based on the user's intent; Based on the task type, a task plan is obtained. The task plan is a structured task list, and each task in the task list specifies the specific processing goal and scope.

3. The method according to claim 2, characterized in that: The processing of the user's current question includes: Based on the user's business knowledge base and pattern matching library, the natural language expression of the user's current question is replaced with a standardized equivalent question.

4. The method according to claim 2, characterized in that: The processing of the user's current question includes: Determine whether there is a time element in the user's current question; If it does not exist, then obtain the candidate preceding question containing time elements and its corresponding receiving time; The time value of the candidate preceding question that was received latest is added to the user's current question.

5. The method according to claim 2, characterized in that: The processing of the user's current question includes: Obtain the dimension values ​​from the user's current question; If a target dimension value corresponds to at least two dimensions, then the target dimension value is retrieved and assembled into a complete sentence list, which is then returned to the user for confirmation.

6. The method according to claim 2, characterized in that, Also includes: After each task is completed, the result status information of the task is obtained. The result status information includes: the completion progress of the task, the details of the processing result, and whether any abnormalities have occurred. Based on the result status information, determine whether a new task needs to be added; If not needed, the task execution results will be integrated to obtain the response data and returned to the user; if needed, a new supplementary task will be generated and added to the task list.

7. An intelligent data query device based on semantic data models and large models, characterized in that, include: The user question receiving module is used to receive the user's current question. The task planning acquisition module is used to obtain the task plan corresponding to the user's current question based on the large model. The MQL information extraction module is used to extract MQL information based on the task planning to obtain the target MQL; The step of extracting MQL information based on the task plan to obtain the target MQL includes: performing metadata retrieval on each task in the task plan to obtain metadata retrieval information; providing prompts, MQL field descriptions, metadata retrieval information, and examples to the large model to obtain the MQL information of the current round with a preset structure; determining whether the MQL information of the current round includes both time elements and indicator elements; if it includes both time elements and indicator elements, obtaining the target MQL based on the MQL information of the current round; if it does not include both time elements and indicator elements, using the corresponding time elements or indicator elements from the historical MQL as the time elements or indicator elements in the MQL information of the current round, and comparing the dimensions included in the current round with the dimensions in the historical MQL; if there are the same dimensions, replacing the historical dimension values ​​with the current dimension values; if it is a new dimension, appending the current dimension value filtering conditions to the historical MQL to obtain the target MQL; The SQL statement conversion module is used to convert the target MQL into an SQL query statement using a semantic data model. The SQL query execution module is used to perform a query based on the SQL query statement to obtain response data and return the response data to the user.

8. An intelligent data query system based on semantic data models and large models, characterized in that, include: At least one processor and at least one memory; The memory stores the executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1-6.

Images