Method for obtaining visual chart through question and answer mode access based on FineBI and Text2DSL

Abstract

The invention relates to the field of BI tools, in particular to a method for obtaining a visual chart through question and answer mode access based on FineBI and Text2DSL. A method for obtaining a visual chart through question and answer mode access based on FineBI and Text2DSL comprises the following steps: step 1, in a user input stage, obtaining user input; 2, determining an index field and / or a dimension field involved in a natural language input by a user; and step 3, generating a DSL structural body in combination with the current question and the previous question of the user. The method has the beneficial effects that the FineBI tool and the Text2DSL technology are combined, so that the capabilities of FineBI data authority management and data query efficiency optimization are reserved, the data consumption threshold of the user is reduced, the data consumption form is improved, and meanwhile, compared with Text2SQL, the method has higher interpretability.

Description

Technical Field

[0001] The present invention relates to the field of BI tools, and in particular to a method for obtaining a visual chart by question-and-answer data extraction based on FineBI and Text2DSL. Background Art

[0002] When there are some data extraction requirements within an enterprise, some BI tools are generally called, such as FineBI, etc. Such tools require reading relevant operation manuals and have a certain operation threshold. On top of such tools, there are also technologies such as Text2SQL to implement data extraction requirements within the enterprise. In addition, there is also Text2DSL technology to implement data extraction requirements.

[0003] The main defects are as follows: (1) Simply extracting data based on BI tools requires a certain operation threshold, and at the same time, the form of data consumption is not as flexible as conversational consumption.

[0004] (2) Based on Text2SQL, the data extraction method that completely separates from BI tools cannot provide information with a high enough interpretability. Operators need to understand the SQL language to verify the validity of the results. At the same time, if the generated SQL statements are not optimized or the data extraction is not accelerated, the query speed is slow. At the same time, it is difficult to control the data permissions without the BI tool.

[0005] (3) Based on Text2DSL, the data extraction method that separates from BI tools also cannot solve the limitations of data permissions and data extraction speed. However, DSL has better interpretability than SQL, which can be used by users to more conveniently check whether the data extraction results are correct. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for obtaining a visual chart by question-and-answer data extraction based on FineBI and Text2DSL to solve the problems raised in the above background art.

[0007] To achieve the above purpose, the present invention provides the following technical solutions: A method for obtaining a visual chart by question-and-answer data extraction based on FineBI and Text2DSL, comprising the following steps: Step 1, user input stage, obtaining user input; Step 2, determining the metric fields and / or dimension fields involved in the natural language of the user input, facilitating the filling of the corresponding metric or dimension fields in the DSL structure generation; Step 3, taking the obtained metric fields and / or dimension fields as input, and at the same time passing the definition of the semantic task and the definition of the DSL structure to the LLM model, and generating the DSL structure in combination with the user's current question and previous questions; Step 4: Convert the identified semantic tasks into data query statements executable by FineBI, where standard structures are used to ensure that the DSL structures output by the model are parsable, executable, and consistent. Step 5: FineBI retrieves data to obtain visual charts.

[0008] As a further solution of the present invention: In the user input stage, input is performed through a preset Q&A interface, where several topics are preset in the Q&A interface; each topic is bound to a data model defined in FineBI. When the user requests to enter the Q&A interface, display the topics to which the user has access rights according to the user's permissions. After the user selects a topic, dynamically obtain the currently available data model through FineBI.

[0009] As a further solution of the present invention: When the user requests to enter the Q&A interface, synchronize the user's data access permissions from FineBI and dynamically filter metric fields and / or dimension fields. Use the data permission API interface of FineBI to only identify and / or match the authorized fields when analyzing metric fields and / or dimension fields of the user's question.

[0010] As a further solution of the present invention: The specific method of Step 2 includes the following steps: Step 2.1: Pre-build a synonym and / or near-synonym configuration file, and use the rule of field synonyms to match metric fields and / or dimension fields. Step 2.2: Calculate the similarity between words based on the edit distance algorithm to further match metric fields and / or dimension fields to make up for text spelling mistakes and long-tail inputs not covered by the rule matching of synonyms. Step 2.3: Perform text vector encoding on the fields and the question, and complete the matching based on cosine similarity through the combination of embedding vectors and similarity matching.

[0011] As a further solution of the present invention: The implementation method of Step 2.1 is as follows: Collect and maintain a synonym and / or near-synonym configuration file of fields, which can be stored in a structured format, perform word segmentation on the user input, and based on the synonym and / or near-synonym configuration file, standardize all matching words into the corresponding field names in FineBI. Among them, the priority of Step 2.1 is higher than that of Step 2.2 and Step 2.3. When there is an explicit hit in Step 2.1, there is no need to perform subsequent Step 2.2 and Step 2.3 for supplementary recognition based on similarity calculation.

[0012] As a further solution of the present invention: In step 2.2, calculate the similarity between words based on the edit distance algorithm, including the following steps: Step 2.2.1, set the candidate field list and the user term list to be matched; Step 2.2.2, calculate the edit distance for any pair of words in the candidate field list and the user term list to be matched; Step 2.2.3, preset the edit distance threshold. If the edit distance ≤ the edit distance threshold, it is considered that the field is hit; Among them, an additional similarity weighting factor is introduced to adjust the matching score through weights, improving the sorting accuracy.

[0013] As a further solution of the present invention: Step 2.3 includes the following steps: Step 2.3.1, construct a semantic representation vector for each field; Step 2.3.2, generate a corresponding question embedding vector for the user question; Step 2.3.3, calculate the cosine similarity between the question embedding vector and each semantic representation vector, and take several fields with the top-ranked cosine similarity as the candidate field pool; Step 2.3.4, perform thresholding and sorting filtering processing, eliminate fields with similarity lower than the set threshold, and return the best field or field set as the input for subsequent semantic task parsing to generate the DSL structure.

[0014] As a further solution of the present invention: For step 3, construct a multi-stage thinking chain prompt template. For different types of tasks, set prompts to enable the model to think in stages and gradually execute the task intention recognition task, and perform soft logical reasoning in stages according to the prompts.

[0015] As a further solution of the present invention: The method of combining the user's current question and the previous question includes the following steps: Preset the context dialogue structure, which includes: several field information; Implement dialogue history pruning based on the memory window or semantic block; For the omitted information in the user's current round of input, call the field information of the context dialogue structure for completion and update.

[0016] As a further solution of the present invention: In step 3, the DSL structure generation strategy: Adopt the method of structure module prompting. For the identified intention structure, let the LLM model map from the semantic intention to the specified DSL structure through the prompt template; In the case where the input intention is not clearly specified, automatically guide the LLM model to generate reasonable default parameters.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: By combining the FineBI tool and the Text2DSL technology, it not only retains the ability of FineBI to manage data permissions and optimize data query efficiency, but also reduces the data consumption threshold for users, improves the form of data consumption, and has higher interpretability compared to Text2SQL.

[0018] Based on the problem of data extraction in complex scenarios within an enterprise, it breaks away from the traditional data consumption form of manual dragging and selection, selects a conversational data consumption form, reduces the data extraction threshold, and combines the permission control and query acceleration optimization capabilities of FineBI to ensure the efficiency and security of data extraction. Thus, users can perform data consumption activities more quickly and effectively.

[0019] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. Description of the Drawings

[0020] Figure 1 is a flowchart of the method for obtaining a visualization chart by means of question-and-answer data extraction based on FineBI and Text2DSL of the present invention. Specific Embodiments

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] As Figure 1 shown, a method for obtaining a visualization chart by means of question-and-answer data extraction based on FineBI and Text2DSL includes the following steps: Step 1, user input stage, obtaining user input; Step 2, determining the index fields and / or dimension fields involved in the natural language of the user input, facilitating the filling of the corresponding index or dimension fields in the DSL structure generation; Step 3, taking the obtained index fields and / or dimension fields as input, and simultaneously passing the definitions of semantic tasks and DSL structures to the LLM model, and generating a DSL structure in combination with the user's current question and previous questions; Step 4, converting the identified semantic tasks into data query statements executable by FineBI, where standard structures are used to ensure that the DSL structures output by the model are parsable, executable, and consistent; Step 5, FineBI retrieves data to obtain a visualization chart.

[0023] As a specific implementation, in the user input stage, the input is made through a preset Q&A interface, where several topics are preset in the Q&A interface; each topic is bound to a data model defined in FineBI; When the user requests to enter the Q&A interface, the topics to which the user has access rights are displayed according to the user's permissions; After the user selects a topic, the currently available data model is dynamically obtained through FineBI.

[0024] As a specific implementation, when the user requests to enter the Q&A interface, the data access permissions of the current user are synchronized from FineBI, and the metric fields and / or dimension fields are dynamically filtered; Using the data permission API interface of FineBI, when analyzing the metric fields and / or dimension fields of the user's question, only the authorized fields are identified and / or matched.

[0025] Specifically, in the user input stage, that is, when the user enters the Q&A interface, selects a topic, and enters a question.

[0026] The user input stage involves: 1. Binding of topic selection and data source; 2. Verification of user identity and permissions.

[0027] 1. Binding of topic selection and data source. Each topic is bound to a data model (i.e., dataset) defined in FineBI. Provide topic previews, involving data dimensions and main metrics. Dynamically obtain the currently available data model from the API port of FineBI. Bind permission information in the background, and the user can only view and select topics to which they have access rights.

[0028] 2. Verification of user identity and permissions can enhance security. When requesting to enter the Q&A interface, the system will synchronize the data access permissions of the current user from FineBI: dynamically filter field permissions (field display / hide); dynamically filter the values on the dimension (such as not showing data of specific stores). Use the data permission API interface of FineBI, such as the entry-level permission interface; when analyzing the fields of the user's question, only the authorized fields are identified and / or matched to avoid overstepping permissions.

[0029] The specific process of user operations includes the following steps: 1. Enter the Q&A interface and select the topic for which you want to conduct Q&A (one topic contains multiple tables); 2. After selecting the topic, you can enter a question for data retrieval and then obtain an answer; 3. The user conducts multiple rounds of data retrieval Q&A as needed; 4. Repeat steps 1 to 3 until the data retrieval result desired by the user is obtained.

[0030] In step 2, field selection is performed to determine the metric fields and / or dimension fields involved in the natural language input by the user, facilitating the filling of the corresponding metric or dimension fields in the generation of the DSL structure. Based on the traditional solution, the present invention further introduces a multi-strategy fusion technology, including synonym normalization, edit distance matching, semantic nesting calculation, and domain word embedding model, etc., to improve the accuracy and robustness of field matching.

[0031] Specifically, the specific method of step 2 includes the following steps: Step 2.1, pre-construct a synonym and / or near-synonym configuration file, and use the rule of field synonyms to match the metric fields and / or dimension fields; Step 2.2, calculate the similarity between words based on the edit distance algorithm, and further match the metric fields and / or dimension fields to make up for text spelling mistakes and long-tail inputs not covered by the rule matching of synonyms; Step 2.3, perform text vector encoding on the fields and questions, and complete the matching based on cosine similarity through the combination of embedding vectors and similarity matching.

[0032] Step 2.1 realizes the unified normalization of the expressions used by users with non-professional terms by constructing a business term dictionary corresponding to the fields, avoiding field matching failures caused by unclear word meanings.

[0033] The implementation method of step 2.1 is as follows: collect and maintain a synonym and / or near-synonym configuration file of the fields, which can be stored in a structured format (such as JSON format or YAML format), perform word segmentation on the user input, and based on the synonym and / or near-synonym configuration file, standardize all matching words into the corresponding field names in FineBI.

[0034] Taking the JSON format as an example, it is as follows: JSON { "Sales amount": ["Total income", "Turnover", "Revenue"], "Store": ["Shop", "Store", "Outlet"] }.

[0035] The priority of step 2.1 is higher than that of step 2.2 and step 2.3. When there is an explicit hit in step 2.1, there is no need to perform subsequent supplementary recognition based on similarity calculation in step 2.2 and step 2.3.

[0036] The edit distance algorithm (Levenshtein algorithm) is adopted to address issues such as user text input errors or spelling deviations. Specifically, a character-level matching model is introduced to calculate the similarity between words based on the edit distance algorithm, which is used to make up for the long-tail inputs not covered by the synonym list configuration.

[0037] Step 2.2: Calculate the similarity between words based on the edit distance algorithm, including the following steps: Step 2.2.1: Set the candidate field list F and the user term list Q to be matched. Step 2.2.2: Calculate the edit distance d for any pair of words in the candidate field list F and the user term list Q to be matched. Step 2.2.3: Preset the edit distance threshold θ. If the edit distance d ≤ the edit distance threshold θ, then it is considered that the field is hit. Among them, a similarity weighting factor is additionally introduced to adjust the matching score through weights to improve the sorting accuracy. For example, the TF-IDF weight is used to adjust the matching score.

[0038] Based on Steps 2.1 and 2.2, a method of combining embedded vector calculation and similarity matching (for example, based on the BERT / BGE model) is further adopted to enhance the semantic recognition ability. Use pre-trained models such as BERT / BGE that support Chinese semantic embedding to perform text vector encoding on the fields and questions, and complete the matching based on cosine similarity.

[0039] Step 2.3 includes the following steps: Step 2.3.1: Construct a semantic representation vector for each field, for example, by calling the sentence-BERT model. Step 2.3.2: Generate a corresponding question embedding vector for the user question. For example, for the user question "Please help me count the income in the past three months", generate the corresponding question embedding vector query_vec. Step 2.3.3: Calculate the cosine similarity between the question embedding vector query_vec and each semantic representation vector, and use the fields with the top several cosine similarity rankings as the candidate field pool. Step 2.3.4: Perform thresholding and sorting and filtering processing, remove the fields with similarity lower than the set threshold, and return the best field or field set as the input for subsequent semantic task parsing to generate the DSL structure.

[0040] Description of semantic tasks: Semantic parsing is to identify which semantic tasks are in the user's question. Specifically, it includes: YoY and MoM: YoY and MoM tasks of indicators, such as YoY of sales. Indicator value filtering: filter the indicator values, for example, sales greater than 1 million; Proportion: the proportion of indicators, for example, the proportion of sales; Sorting: The task of sorting / ranking indicators, for example, the sales ranking of Shanghai; Count: Dimensional counting tasks, for example, how many stores have sales greater than 1 million this year?

[0041] Explanation of the definition of DSL structure: Specific examples such as: date_list: an array of time range descriptions in the user's question, e.g., this year, last year; timeunit_list: an array of time unit descriptions in the user's question. Supported time units include monthly, weekly, monthly, weekly, and daily. dimension_list: an array of table dimension fields in the user's question, such as city, province, store; dimension_filter_list: An array of dimension field enumeration values in the user's question. The enumeration value should be strictly consistent with the user's question, unless the reference document has relevant reference content, such as Shanghai; useful_info_list: The useful information in the user's question is arranged in the order in which it appears in the user's question. Only the time range description, time unit, indicator, dimension and dimension enumeration value are considered, for example, ["this year", "month", "Shanghai", "sales"]; count_tasks: An array of dimension counting tasks in the user's problem. Each task is defined by the following fields: dimension: the dimension to be counted; sort_tasks: An array of tasks for sorting metrics in the user's problem. Each task is defined by the following fields: metric: the specific metric name to be sorted; sort_type: sorting method, DESC means descending order, ASC means ascending order; calculation_type: calculation type. Supported calculation types include: PROP, which indicates percentage calculation; YoY, which indicates year-on-year calculation; and MoM, which indicates month-on-month calculation. This parameter can be empty. aggregation_function: aggregation method. Supported functions include SUM and AVG. The default function is SUM. group_by_dimensions: An array of field names for grouping dimensions, used for aggregation calculations, can be empty; start_of_row: From which row of the data to start returning results (counting from 1); end_of_row: The ending row number to return.

[0042] fliter_tasks: An array of metric value filtering tasks in the user's question. Each task is defined by the following fields: metric: The specific metric name to be filtered; aggregation_function: Aggregation method, supported functions include SUM and AVG, default SUM; calculation_type: Calculation type, supported calculation types are: PROP for percentage calculation; YoY for year-on-year calculation; MoM for month-on-month calculation. Can be empty; group_by_dimensions: An array of field names for grouping dimensions, used for aggregation calculations, can be empty; operator: Filter operator, supported operators are LE, GE, LT, GT, Between; values: Filter values, e.g.,

[1000] , [1000, 3000].

[0043] calculation_list: An array of metric calculations in the user's question, not involving filtering and sorting. Each task is defined by the following fields: metric: The specific metric name to be filtered; aggregation_function: Aggregation method, supported functions include SUM function and AVG function, default SUM function.

[0044] calculation_type: Calculation type, supported calculation types are: PROP for percentage calculation; YoY for year-on-year calculation; MoM for month-on-month calculation. Can be empty.

[0045] Take the obtained metric fields and / or dimension fields as input, and at the same time pass the definition of the semantic task and the definition of the DSL structure to the LLM model, and generate the DSL structure in combination with the user's current question and the previous question.

[0046] Semantic parsing is a crucial step in converting the natural language input by users into a structured query recognizable by machines. Its core lies in identifying the semantic task types and their corresponding parameter elements contained in the user's query intention, providing a structural basis for the generation of DSL structures. This invention uses a large language model (LLM model, Large Language Model) to complete this step. Compared with traditional rule-based methods, this solution has stronger language understanding ability and context modeling ability, and can effectively identify complex semantic nesting and multi-task fusion problems.

[0047] To further improve the task recognition accuracy and semantic integrity of the large language model (LLM model) in processing complex, multi-level, and multi-round inputs, this invention introduces a series of enhanced language understanding mechanisms in the basic solution to improve the recognition stability and logical consistency in the process of converting natural language to DSL structures, specifically including: Introduce inference-enhanced prompt templates (such as ReAct Prompt template, Chain-of-Thought Prompt template); Multi-round conversation state management mechanism (including semantic window memory, dialogue state container); User query rewriting mechanism (Query Rewriting problem rewriting); Through the above technical applications, the semantic structure of the natural language input is disassembled into a computable task list, and it is ensured that the task boundaries are clear, the parameter extraction is complete, and the reasoning process is transparent.

[0048] For step 3, construct a multi-stage thinking chain prompt template. For different types of tasks, set prompts to enable the model to think in stages and gradually execute the task intention recognition task, and perform soft logical reasoning in stages according to the prompts.

[0049] To improve the stability of the large language model in processing multi-intentions, nested structures, and ambiguous sentences, this invention adopts a semantic prompt strategy that is completely different from the traditional "single-round structured Prompt prompt". This method uses a Prompt template with the ability of "reasoning chain" to establish the thinking step structure inside the model, thereby improving the accuracy of semantic recognition and task disassembly.

[0050] Specific implementation steps: 1. Construct a multi-stage thinking chain prompt template. For different types of tasks (such as year-on-year calculation + sorting + proportion analysis), design the Prompt prompt content to enable the model to think in stages:

User question

Goal

[0051] 2. Reuse the Prompt template to support multiple question variants This Prompt template can be used in multiple question scenarios, supporting semantic normalization of different expressions (such as "this year", "last year", "the top five", "5 cities", etc.). In the Few-shot Prompting mode, several examples can be added to improve the generalization ability of the model.

[0052] 3. Gradually execute the task intention recognition task The model does not need to complete all structure extractions at once, but performs soft logical reasoning in stages according to the prompts. For example: Step1: Confirm the metric: Sales Step2: Include year-on-year: Yes, the comparison time is last year Step3: Include sorting requirements: Yes, rank according to the growth rate Step4: Include percentage requirements: Yes, the object is the total annual sales This chain of thought not only improves the accuracy of generating the DSL structure but also provides an interpretable analysis trace for subsequent debugging.

[0053] To handle the elliptical, referential, or context-dependent questions (such as "One more for last month") raised by users in multi-round Q&A, the present invention designs a multi-round semantic context management mechanism for "semantic merging" of the questions raised by users in each round with the historical semantic state.

[0054] The method of combining the user's current question and the previous question, that is, the method of semantic merging, includes the following steps: Preset a context dialogue structure (Dialog Context structure), which includes: several field information; Implement dialogue history trimming based on the memory window or semantic chunks; For the omitted information in the user's current round of input, call the field information of the context dialogue structure for completion and update.

[0055] The fields of the Dialog Context structure include: JSON { "last_query": "The top five stores in terms of this year's sales", "active_dimensions": ["store"], "active_metrics": ["sales"], "active_time_range": ["this year"], "previous_filters": {"store": ["Beijing", "Shanghai"]} }

[0056] Dialogue history pruning based on Memory Window or Semantic Chunk. To prevent the model response from being overloaded due to context backlog, the system adopts an N-round window control mechanism (for example, N is set to 3, the last 3 rounds of history), or performs similarity merging on the semantic state vectors of each round to compress the information volume.

[0057] If the user inputs omitted information in the current round (such as "What about the performance last month?"), the system will automatically reuse the previous active_dimensions field and active_metrics field, and only update active_time_range = ["last month"].

[0058] The finally synthesized question is as follows:

Full question after synthesis

[0059] Introduce the Query Rewriting technology. For problems such as semantic ambiguity, syntactic chaos, or logical incompleteness in the user input, in order to improve the generation effect of the downstream DSL structure, the present invention introduces a Query Rewriting question cleaning mechanism for semantic normalization and rewriting of the user's original question before the large model parsing.

[0060] The specific steps are as follows: (1) Introduce a dedicated Query Rewriting model The system integrates a fine-tuned language model based on Transformer (which can be fine-tuned through basic models such as T5 / Bart), specifically for "semantic transformation" of the user's original query question: Input: What is the performance of stores with sales of over 500,000 this year? Output: List the sales amounts of each store with this year's sales exceeding 500,000 yuan.

[0061] (2) Rewriting model combined with intent classification and rule correction module The model can trigger the Rewrite action after intent recognition in combination with the Intent-classifier model. For example, a syntactic structure like "how... in..." is determined to be a semantically incomplete sentence pattern.

[0062] (3) Return a standard question after rewriting and record the original question The system retains both the original Query question and the Rewritten Query question for model invocation and question backtracking. Ultimately, the cleaned rewritten question is used for DSL structure generation, but the original question is still retained in the system interface for users to reference and confirm.

[0063] Semantic task recognition process for specific questions After completing the context information supplementation for the user input question, the corresponding semantic task will be recognized.

[0064] Semantic tasks include but are not limited to: year-on-year (YoY), month-on-month (MoM), indicator proportion (PROP), indicator value filtering, indicator sorting, dimension counting, time trend analysis, etc.

[0065] Semantic task recognition and structure extraction (driven by the LLM model) The semantic recognition module takes the large language model as the core and constructs a structured recognition prompt template to let the model output a structured semantic task list.

[0066] Core capabilities include: category recognition ability, parameter extraction ability, multi-intent decomposition ability, etc.

[0067] Category recognition ability: The model can recognize key prompt words with semantic orientation in natural language. For example: "Year-on-year", "same period last year" ==> trigger year-on-year (YoY); "Month-on-month", "last month" ==> trigger month-on-month (MoM); "Ranking", "highest", "top ten" ==> trigger sorting task; "Account for", "proportion" ==> trigger proportion task; "How many" ==> trigger counting task; "Greater than", "more than", "not less than" ==> numerical filtering task.

[0068] Parameter extraction ability: While identifying semantic tasks, the model extracts relevant metrics (such as "sales"), time ranges (such as "this year", "the past three months"), dimension fields (such as "city", "store"), dimension values (such as "Shanghai"), etc.

[0069] Multi-intention decomposition ability: If the user's question contains multiple sub-intentions, such as "Tell me the proportion of sales in the top five cities with the highest year-on-year growth in Shanghai this year", the model can automatically identify and decompose it into: Year-on-year calculation of indicators; Sorting based on year-on-year changes; Select the top five; Calculate the proportion of the sales of each of the top five cities; And synchronously extract the corresponding entity fields and time dimensions.

[0070] Finally, the LLM model inputs the following standard JSON format result with a preset connector for the subsequent code generation of the DSL structure: JSON { "Task type": ["Year-on-year", "Sorting", "Proportion"], "Time range": ["This year"], "Time unit": ["month"], "Indicator": ["Sales"], "Dimension": ["City"], "Dimension enumeration": ["Shanghai"], "Sorting method": "DESC", "TopN": 5 }。

[0071] This structure is saved by this system in an intermediate state and will directly participate in the scheduling of the DSL structure logic in the next stage.

[0072] The large language model adopted has the following significant advantages in semantic parsing: Strong context understanding ability: It can perform semantic connection based on the previous and next rounds of questions, complete question disambiguation and even supplement omitted information; Compatibility with rich language expressions: Users can express task intentions in various forms, such as "top five", "top 5", "Top5", and the large model can recognize them all; Outstanding structure mapping ability: Without having to manually write a large number of rules, the model can complete the mapping from natural language to structure representation by itself; Strong complex semantic parsing ability: It supports the recognition and splitting of nested intentions and multi-task combination scenarios; Strong sustainable optimization ability: Continuously iterate and optimize the model prompts by mixing Few-shot model prompts and posterior evaluation results.

[0073] In step 3, the DSL structure generation strategy: Adopt the method of structure module prompts. For the identified intent structure, use the prompt template to let the LLM model map from the semantic intent to the specified DSL structure; In the case where the input intent is not clearly specified, automatically guide the LLM model to generate reasonable default parameters.

[0074] A set of DSL structure assembly rules for converting various identified semantic tasks into executable data query statements in FineBI, where standard structures are used to ensure the parsability, executability, and consistency of the DSL structures output by the model.

[0075] DSL stands for "Domain Specific Language Data Query Description Language". Its essence is a JSON structure that describes: Query time (date_list / timeunit_list); Query dimensions and filter values (dimension_list / dimension_filter_list); Query task types (sorting, filtering, calculation, counting, percentage, etc.).

[0076] For the identified intent structure, use the prompt template to let the LLM map from the semantic intent to the specified DSL structure. For example: User intent structure: { "task_type": ["sorting", "year-on-year"], "time_range": ["this year"], "dimension": ["city"], "metric": ["sales"], "sorting_method": "DESC", "TopN": 5 } Please generate a structure that conforms to the DSL specification based on the following fields: ... Output (DSL structure generated by the model): JSON { "date_list": ["this year"], "dimension_list": ["city"], "useful_info_list": ["this year", "city", "sales amount"], "sort_tasks": { "metric": "sales amount", "sort_type": "DESC", "calculation_type": "YoY", "aggregation_function": "SUM", "group_by_dimensions": ["city"], "start_of_row": 1, "end_of_row": 5 } }。

[0077] In the case where the input intention is not clearly specified, the system will automatically guide the model to generate reasonable default parameters. For example: If there is no aggregation_function, the default is "SUM function" If start / end_row is not specified but "top N" exists, the system will parse it as a TopN requirement and automatically fill it in. N is a natural number, and TopN refers to the top N data in the ranking; If phrases such as "each city" and "different stores" are included, the dimension field group_by_dimensions can be inferred.

[0078] Let FineBI retrieve data based on the received DSL structure to obtain a visualization chart.

[0079] After inputting the DSL structure, FineBI converts the DSL structure into a general chart component of FineBI, and then retrieves data through the chart component. The generated general component can be further modified in FineBI, and users can fine-tune the chart component to meet more diverse data retrieval requirements. The data retrieval process of the chart component in FineBl will be accelerated through the self-developed engine of FineBI to meet the needs of users for rapid data retrieval under complex queries.

[0080] In the synonym configuration, configure synonyms for each field; configure synonyms for dimension enumeration values; customize industry terms; the administrator can monitor and view conversation records, user feedback, etc. in real time.

[0081] ​In intelligent Q&A, topics / topic sets can be selected on the left side of the intelligent Q&A interface to determine the counting range; recommended questions can be selected in the middle of the interface for quick query; users can enter questions in the input box, and can select to count, ask for ideas, or ask the knowledge base, and can control whether to inherit multiple rounds.

[0082] Interpret the data and analyze the problem through thinking.

[0083] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claimed invention.

[0084] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for obtaining a visual chart by query-based data extraction based on FineBI and Text2DSL, characterized in that It includes the following steps: Step 1, user input stage, obtaining user input; Step 2, determining the metric fields and / or dimension fields involved in the natural language of the user input, facilitating the filling of corresponding metric or dimension fields during DSL structure generation; Step 3, taking the obtained metric fields and / or dimension fields as input, and simultaneously passing the definitions of semantic tasks and DSL structures to the LLM model, and generating the DSL structure in combination with the user's current question and previous questions; Step 4, converting the identified semantic tasks into data query statements executable by FineBI, where standard structures are used to ensure the parseability, executability, and consistency of the DSL structure output by the model; Step 5, FineBI retrieves data to obtain a visualization chart.

2. The method for obtaining a visualization chart through question-and-answer data retrieval based on FineBI and Text2DSL according to claim 1, wherein the user input stage is input through a preset question-and-answer interface, wherein several topics are preset in the question-and-answer interface; each of the topics is bound to a data model defined in FineBI; when the user requests to enter the question-and-answer interface, the topics to which the user has access rights are displayed according to the user's permissions; after the user selects a topic, the currently available data model is dynamically obtained through FineBI.

3. The method for obtaining a visualization chart through question-and-answer data retrieval based on FineBI and Text2DSL according to claim 2, wherein when the user requests to enter the question-and-answer interface, the data access permissions of the current user are synchronized from FineBI, and the metric fields and / or dimension fields are dynamically filtered; using the data permission API interface of FineBI, when analyzing the metric fields and / or dimension fields of the user's question, only the authorized fields are identified and / or matched.

4. The method for obtaining a visualization chart through question-and-answer data retrieval based on FineBI and Text2DSL according to claim 1, wherein the specific method of step 2 includes the following steps: Step 2.1, pre-constructing a synonym and / or near-synonym configuration file, and matching the metric fields and / or dimension fields according to the rules of field synonyms; Step 2.2, calculating the similarity between words based on the edit distance algorithm, and further matching the metric fields and / or dimension fields to make up for text spelling mistakes and long-tail inputs not covered by the rules of synonym matching; Step 2.3, performing text vector encoding on the fields and questions, and completing the matching based on cosine similarity through the combination of embedding vectors and similarity matching.

5. The method for obtaining a visualization chart through question-and-answer data retrieval based on FineBI and Text2DSL according to claim 4, wherein the implementation method of step 2.1 is: collecting and maintaining a synonym and / or near-synonym configuration file of fields, which can be stored in a structured format, performing word segmentation on the user input, and based on the synonym and / or near-synonym configuration file, standardizing all matching words into the corresponding field names in FineBI; Among them, the priority of step 2.1 is higher than that of step 2.2 and step 2.

3. When there is an explicit hit in step 2.1, there is no need to perform subsequent supplementary recognition based on similarity calculation in step 2.2 and step 2.

3.

6. The method for obtaining a visual chart through question-and-answer data extraction based on FineBI and Text2DSL according to claim 4, wherein Step 2.2, calculating the similarity between words based on the edit distance algorithm, includes the following steps: Step 2.2.1, setting a candidate field list and a list of user terms to be matched; Step 2.2.2, calculating the edit distance for any pair of words in the candidate field list and the list of user terms to be matched; Step 2.2.3, presetting an edit distance threshold. If the edit distance ≤ the edit distance threshold, it is considered that the field is hit; Among them, a similarity weighting factor is additionally introduced to adjust the matching score through weights to improve the sorting accuracy.

7. The method for obtaining a visual chart through question-and-answer data extraction based on FineBI and Text2DSL according to claim 4, wherein Step 2.3, includes the following steps: Step 2.3.1, constructing a semantic representation vector for each field; Step 2.3.2, generating a corresponding question embedding vector for the user's question; Step 2.3.3, calculating the cosine similarity between the question embedding vector and each semantic representation vector, and taking several fields with the top-ranked cosine similarity as the candidate field pool; Step 2.3.4, performing thresholding and sorting filtering processing, removing fields with similarity lower than the set threshold, and returning the best field or field set as the input for subsequent semantic task parsing to generate the DSL structure.

8. The method for obtaining a visual chart through question-and-answer data extraction based on FineBI and Text2DSL according to claim 1, wherein For step 3, construct a multi-stage thinking chain prompt template. For different types of tasks, set prompts to enable the model to think in stages and gradually execute the task intention recognition task, and perform soft logical reasoning in stages according to the prompts.

9. The method for obtaining a visual chart through question-and-answer data extraction based on FineBI and Text2DSL according to claim 1, wherein The method of combining the user's current question and the previous question includes the following steps: Preset a context dialogue structure, which includes: several field information; Implement dialogue history pruning based on a memory window or semantic chunks; For the omitted information input by the user in the current round, call the field information in the context dialogue structure for completion and update.

10. The method for obtaining a visual chart through question-and-answer data extraction based on FineBI and Text2DSL according to claim 1, wherein In step 3, the DSL structure generation strategy: Adopt the method of structure module prompting. For the identified intention structure, let the LLM model map from the semantic intention to the specified DSL structure through the prompt template; In the case where the input intention is not clearly specified, automatically guide the LLM model to generate reasonable default parameters.

Images