Conversational business intelligent platform process optimization method, system and equipment

Abstract

The invention belongs to the technical field of artificial intelligence, and provides a dialogue type business intelligent platform process optimization method, system and equipment in order to solve the problems of random errors, result adaptability defects and the like existing in a ChatBI process, and output results of all links in the dialogue type business intelligent platform process are obtained. Multi-dimensional verification is carried out on output results of all links in sequence according to the process execution sequence, the process links which possibly have problems can be found in time, and correction and optimization are carried out in time before errors are diffused to the next process link; and the output result is verified from three aspects of adapting to the original demand of the user input problem, conforming to the business logic of the process link and adapting to the use demand of the downstream process link, so that the suitability of the result is ensured, and the problems that each process link has no obvious error, but the output of the process link is disjointed from the downstream process link are avoided.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field related to artificial intelligence, and particularly relates to a conversational business intelligence platform process optimization method, system and device. BACKGROUND

[0002] The statements in this section merely provide background information related to the application and do not necessarily constitute prior art.

[0003] With the deepening of digital transformation, the demand of enterprises for data value mining is increasingly urgent. As a core tool for supporting data-driven decision-making, the function evolution and technology upgrading of business intelligence BI always focus on "reducing the data use threshold" and "improving analysis efficiency". In this context, conversational analysis such as ChatBI gradually becomes the mainstream way for enterprise users to obtain data insights, as it provides the convenience of natural language interaction. Users do not need to master complex SQL syntax or BI tool operation logic, but can initiate data queries and generate visual charts through daily language, greatly expanding the coverage of data applications.

[0004] The core function of the conversational business intelligence platform, ChatBI, is to use large language models and retrieval enhancement generation technology based on domain knowledge base to respond to users' data analysis needs. To achieve accurate conversion from natural language to data charts, ChatBI needs to complete multiple complex tasks, including natural language parsing, intent recognition, data retrieval, query statement generation, and visualization rendering. Since a single agent cannot balance the professionalism and efficiency of all aspects, a multi-agent collaborative architecture has gradually become the mainstream design idea: through a word segmentation agent to process language structure, a Rag service to call knowledge base information, a model selection agent to match the optimal processing model, a question understanding agent to analyze users' real needs, and an NL2SQL agent to generate data query statements, each module cooperates to form a complete processing link.

[0005] However, the multi-agent collaborative mode, while improving system flexibility, also introduces new technical challenges, specifically: 1. There is a risk of "random error" in the processing results of each link. For example, large models may hallucinate during intent understanding or SQL generation, i.e., generate content that does not match the facts, and the enhanced retrieval service may have low information similarity when retrieving relevant data due to limitations of the matching algorithm. These errors, if not discovered in time, will be passed down the processing chain, forming a "error accumulation effect", i.e., the deviation of one link may cause all subsequent steps to run based on incorrect premises, ultimately leading to a serious disconnection between the generated charts and user needs. Since the entire process is highly automated, real-time human monitoring cannot cover all details, greatly increasing the probability of losing control of the results. 2. Even if each intelligent agent operates normally according to the preset logic, result adaptability defects may still occur. For example, when a user queries "quarterly revenue comparison of each department", the system may only return the original association data of department ID and revenue value, without considering the rationality of visual presentation such as lack of structured processing of time dimension, resulting in that the generated chart cannot intuitively reflect the trend and loses the actual analysis value.

[0006] The above problems directly restrict the reliability and user trust of the ChatBI system. Enterprise users may reduce the willingness to use due to frequent result deviations, and even give up relying on system decision-making, hindering the effective release of data value. Therefore, how to solve the problems of random errors and result adaptability defects in the ChatBI process of multiple intelligent agents has become a key direction for current technical development to break through. SUMMARY

[0007] To overcome the shortcomings of the prior art, the present application provides a chat-based business intelligence platform process optimization method, system and device, which can timely find possible error process links, ensure the adaptability of the results, and avoid the problem that the output of each process link is disconnected from the downstream process link.

[0008] To achieve the above purpose, the present application adopts the following technical solutions: In a first aspect, the present application provides a chat-based business intelligence platform process optimization method, comprising: obtaining the output results of each link in the chat-based business intelligence platform process, and sequentially performing multi-dimensional verification on the output results of each link according to the process execution order to determine whether the output results of the process link meet the output requirements; wherein the output requirements include adaptation to the original demand of the user input question, compliance with the business logic of the process link itself, and adaptation to the use demand of the downstream process link; According to the verification results of the output results of each link, a standardized problem identifier is generated, and the corresponding error type and matching processing mode are determined by combining the pre-set keyword library with the standardized problem identifier, so as to realize the correction and optimization of the corresponding process link.

[0009] In a second aspect, the present application provides a chat-based business intelligence platform process optimization system, comprising: The verification module is configured to: obtain the output results of each link in the chat-based business intelligence platform process, and sequentially perform multi-dimensional verification on the output results of each link according to the process execution order to determine whether the output results of the process link meet the output requirements; wherein the output requirements include adaptation to the original demand of the user input question, compliance with the business logic of the process link itself, and adaptation to the use demand of the downstream process link; The optimization module is configured to generate a standardized problem identifier according to a check result of the output result of each link, combine a preset keyword library with the standardized problem identifier as an index, determine a corresponding error type and a matching processing mode, and realize correction and optimization of the corresponding process link.

[0010] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the method of the first aspect is completed.

[0011] The above one or more technical solutions have the following beneficial effects: In the present application, the output results of each link in the conversational business intelligence platform process are obtained, and the output results of each link are checked in sequence according to the process execution sequence, so that the process link that may have problems can be found in time, and the optimization is corrected in time before the error spreads to the next process link; and the output results are checked from three aspects of adaptation to the original demand of the user input question, compliance with the business logic of the process link itself and adaptation to the use demand of the downstream process link, so as to ensure the adaptability of the results and avoid the problem that the output of each process link is not obviously wrong, but the output of the process link is out of touch with the downstream process link.

[0012] The advantages of the additional aspects of the present application will be partially given in the following description, partially become obvious from the following description, or be known by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which form a part of the present application, are used to provide further understanding of the present application, and the illustrative embodiments of the present application and their description are used to explain the present application, and do not constitute improper limitations on the present application.

[0014] Fig. 1 The flowchart of the conversational business intelligence platform process optimization method in the embodiment one of the present application is shown in the figure. Fig. 2 The block diagram of the conversational business intelligence platform process optimization method in the embodiment one of the present application is shown in the figure. DETAILED DESCRIPTION

[0015] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0016] It should be noted that the terms used herein are only for the purpose of describing the specific embodiments, and are not intended to limit the exemplary embodiments according to the present application.

[0017] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0018] Embodiment one The embodiment discloses a dialogue business intelligence platform process optimization method, comprising: The output results of each link in the dialogue business intelligence platform process are acquired, and the output results of each link are sequentially subjected to multi-dimensional verification according to the process execution sequence, so as to determine whether the output results of the process link meet the output requirements; wherein the output requirements include adaptation to the original requirement of the user input question, compliance with the business logic of the process link itself and adaptation to the use requirement of the downstream process link. According to the verification results of the output results of each link, a standardized question identifier is generated, and the corresponding error type and matching processing mode are determined by combining the preset keyword library with the standardized question identifier as an index, so as to realize the correction and optimization of the corresponding process link.

[0019] The embodiment acquires the output results of each link in the dialogue business intelligence platform process, and sequentially subjects the output results of each link to multi-dimensional verification according to the process execution sequence, so that the process link that may have problems can be found in time, and the correction and optimization can be made in time before the error spreads to the next process link; and the output results are verified from three aspects of adaptation to the original requirement of the user input question, compliance with the business logic of the process link itself and adaptation to the use requirement of the downstream process link, so as to ensure the adaptability of the results and avoid the problem that each process link has no obvious error, but the output of the process link is out of line with the downstream process link.

[0020] The dialogue business intelligence platform, namely ChatBI, mainly includes four core processes, namely entity extraction and enhanced retrieval service link, model selection link, question understanding link and execution SQL link.

[0021] The entity extraction link mainly extracts keywords from the natural language question input by the user using the semantic understanding ability of the intelligent agent, and performs character similarity matching retrieval on the keywords and structured information such as model fields and enumeration values; meanwhile, the enhanced retrieval service calling vector library recalls the model key description information and business knowledge, and outputs the range of alternative models corresponding to the user question and the supplementary knowledge system.

[0022] The model selection link focuses on accurate matching of models. Model-related information is summarized from the recalled knowledge, and prompt words are extracted, and 1 to multiple models most suitable for the current user question are screened out by means of the advanced language understanding ability of the large model. These screened models will serve as the core analysis carrier of the subsequent answer process, laying a foundation for accurately answering the user question.

[0023] The problem understanding link combines model field enumeration, model description and other multi-dimensional information to simulate the logical disassembly method of professional data analysts and comprehensively and accurately understand the user's problem. The core output includes multiple key detail information, specifically covering standardized query problem definition, query intent interpretation, selected model and field, and whether data aggregation operation is needed, etc., to provide guidance for SQL generation.

[0024] The SQL execution link automatically generates the target SQL statement and performs the corresponding query operation after receiving the result output by the problem understanding intelligent service, combining the supplementary knowledge obtained by the part-of-speech tagging and Rag enhanced retrieval process, and combining the rules given by the prompt word. Finally, the query result is returned to the front end for display, thereby completing the complete answer to the user's problem.

[0025] The embodiment optimizes the process link on the basis of the existing ChatBI, and the following will be described in combination with Figs. 1-2 A process optimization method of a chat-based business intelligence platform is described in detail, which specifically includes: Step 1: Obtain the output results of each link in the chat-based business intelligence platform process, and sequentially perform multi-dimensional verification on the output results of each link according to the process execution order to determine whether the output results of the process link meet the output requirements; wherein the output requirements include adaptation to the original demand of the user input problem, compliance with the business logic of the process link itself and adaptation to the use demand of the downstream process link.

[0026] In the embodiment, the running states and output results of the entity extraction link, the model selection link, the problem understanding link, the NL2SQL link and the execution SQL link in the chat-based business intelligence platform process are obtained; in combination with the running states of the corresponding process links, the output results of the entity extraction link, the model selection link, the problem understanding link, the NL2SQL link and the execution SQL link are sequentially verified in multiple dimensions to determine whether the output results of the process link meet the output requirements.

[0027] The embodiment takes the user input problem entering the chat-based business intelligence platform as the starting point, and around the key links of entity extraction, model selection, problem understanding, NL2SQL and execution SQL, sequentially develops monitoring, verification, decision intervention and optimization feedback, forms a monitoring-verification-intervention-optimization closed loop, and guarantees the reliability and accuracy of the system in processing user problems.

[0028] After the user completes the selection of the theme and the input of the problem operation, the process link is entered, and the running states and output results of the intelligent agents such as entity extraction, model selection, problem understanding, NL2SQL and execution SQL are captured in real time, and the data tracking link of each link is established.

[0029] In this embodiment, the output result of the entity extraction link is checked in multiple dimensions in combination with the running state of the entity extraction link. Specifically, if the running state of the entity extraction link is keyword matching success, it is determined whether the entity extraction result output by the current entity extraction link is aligned with the semantic of the user input question through character similarity matching and core element comparison; it is determined whether the candidate model recalled by the enhanced retrieval service based on the extracted entity belongs to a model type strongly associated with the extracted entity, and whether the business knowledge recalled by the enhanced retrieval service is consistent with the semantic of the extracted entity; based on semantic similarity, a historical similar question matching the user input question is determined, and the difference between the standard entity of the historical similar question and the extracted entity of the user input question is calculated to check the rationality of the entity extraction result.

[0030] In this embodiment, the checking of the entity extraction link includes comparing the matching degree of the user input question and the entity extraction result (i.e., the keyword list), and the candidate model and business knowledge output by the RAG (i.e., the strong retrieval service), checking whether the model keywords are accurately identified, and determining whether there is missing of key information or incorrect identification. Exemplarily, the keywords include but are not limited to model fields, enumerated values, business indicators, time information, dimension information, etc.

[0031] As a specific implementation, the core element types in the user's original question are analyzed, and the coverage of the element types of the entity extraction result is compared. If any core element type is missing, it is marked as “incomplete entity”. The core element types include but are not limited to business indicators, time dimensions, subject objects, analysis dimensions, etc. The extracted entity value is calculated for character similarity with the model fields and enumerated values in the user input question. If the similarity is lower than a preset threshold, or the entity value is not within the range of the model fields and enumerated values in the user input question, it is marked as “incorrect entity identification”.

[0032] Exemplarily, when querying the revenue of a digital enterprise, the revenue or operating income is not extracted, and the entity is marked as “incomplete entity”.

[0033] The similarity calculation can use one of the cosine similarity, edit distance, etc.

[0034] As a specific implementation, after the verification entity extraction result participates in recall, the RAG checks whether the candidate model recalled based on the extracted entity is of a model type strongly associated with the entity, realizes model matching verification, checks whether the business knowledge recalled by the RAG is consistent with the semantics of the extracted entity, realizes business knowledge verification, if the proportion of knowledge irrelevant to the extracted entity in the RAG recall result exceeds a preset threshold, such as extracting 3 core entities but recalling 10 irrelevant knowledge, then mark "RAG recall redundancy", if the number of recalled knowledge is lower than the preset threshold, such as extracting 3 core entities and only recalling 1 relevant knowledge, then mark "RAG recall deficiency".

[0035] For example, the entity extraction result is "operating income", if the RAG recalls "profit table model V3" and "operating income calculation rule", the verification passes, if it recalls "balance sheet model V2" and "liability field definition", it is marked as "RAG model recall error, business knowledge recall error".

[0036] As a specific implementation, taking the user input question as the retrieval condition, in the historical question library of the RAG knowledge base, the top X similar historical questions in semantics are recalled, the standard entities of these historical similar questions are extracted, the difference degree between the current entity extraction result and the historical standard entity is calculated, if the difference degree exceeds a preset threshold such as lacking more than one core entity, then trigger secondary verification, re-call the entity extraction agent, and based on the element type prompt of the historical standard entity, supplement the missing entity. For the secondary extracted entity result, perform semantic matching verification and RAG result verification again, if it still cannot achieve the difference degree ≤ preset threshold with the historical standard entity, then finally mark "entity extraction exception".

[0037] In this embodiment, through semantic matching and RAG cross verification in the entity extraction link, it avoids that the missing / incorrect keywords and RAG recall biased knowledge are passed to the downstream.

[0038] In this embodiment, the output result of the model selection link is verified in multiple dimensions in combination with the running state of the model selection link, specifically: if the running state of the model selection link is model matching success, it is judged whether the theme label of the business description of the selected model output by the current model selection link is consistent with the theme label of the user input question; wherein the theme label is used to indicate the business scenario; based on the user input question and the corresponding theme label matching, the historical similar questions are obtained, and the model selection result of the historical similar questions is used to verify the model selection result of the current user input question; based on the entity extraction result of the entity extraction link and the business knowledge of the enhanced retrieval service, the core field type required by the user input question is analyzed, and it is verified whether the field list of the selected model covers the core field type required by the user input question.

[0039] In this embodiment, the adaptability of the candidate model service description to the current user problem scenario is verified, and the historical model calling effect data is combined to evaluate the rationality of model selection.

[0040] As a specific implementation, a special intelligent agent is called to classify the business description of the selected model and the user input question by subject label respectively. If the subject labels of the two are inconsistent, it is directly marked as "model mismatch"; if the subject labels are consistent, the matching degree of the core demand of the model business description and the user input question is further calculated by a semantic similarity algorithm. If the matching degree is lower than a preset threshold such as 80%, it is still marked as "insufficient scene adaptability".

[0041] For example, the business description of the selected model is that the model document V4 is used for inventory turnover rate, inventory stock, and inventory age analysis; the subject labels are, for example, sales analysis, inventory management, financial accounting, and human statistics.

[0042] For example, the user input question is of the "sales analysis" theme, and the selected model is of the "inventory management" theme, which is marked as "model mismatch".

[0043] As a specific implementation, the user input question core keyword and the corresponding subject label are taken as retrieval conditions to recall TOPX similar problem records from the historical problem library, and the model selection results, answer accuracy and user feedback in the similar problem records are extracted. If the average answer accuracy of the currently selected model in similar problems is lower than a preset standard, such as lower than 85%, or there are multiple records of user feedback results being invalid, it is marked as "model historical effect not meeting the standard". At the same time, if there is a better model in the historical similar problems, but the current one is not selected, the reason for the current model selection needs to be verified for reasonableness. If the reason is not sufficient, it is marked as "model selection reason invalid".

[0044] For example, in the historical similar "inventory turnover rate query" problem, the average accuracy of "inventory model V4" is 92%, and the average accuracy of "inventory model V3" is 75%. If the currently selected "inventory model V3" has no reasonable reason, it is marked as "model historical effect not meeting the standard, selection reason invalid".

[0045] Based on the keywords and RAG business knowledge output by the previous entity extraction link, the core field types required by the user question are analyzed, including time dimension field, analysis dimension field, core indicator field, and filtering condition field. The field list of the selected model is queried to check whether it completely covers the user demand field types. If any core field is missing, it is marked as "field missing"; if the data types of the model field and the user demand field are not compatible, it is marked as "field type mismatch".

[0046] For example, if the user input question requires the "department name" dimension for chart display, and the selected model only contains the "department ID" field, mark "missing field"; if the model "inventory turnover rate" field is of string type and cannot support numerical sorting, mark "field type mismatch".

[0047] As an optional implementation, the structured entity list of the user input question is extracted, the field types required by the user input question are preliminarily screened based on the mapping rules of the entity types and the core field types, the field types required by the user input question are completed and corrected based on the business knowledge of the enhanced search service, and the core field types required by the user input question are obtained; wherein the business knowledge of the enhanced search service includes the mapping relationship between the entities and the fields associated with the user input question, and the field types.

[0048] Specifically, the structured entity list output by the entity extraction link is traversed, the field types required by the user question are preliminarily screened according to the mapping rules of the entity types and the core field types; if there is a field type that is implied by the user question but not covered by the entity extraction in the RAG business knowledge, the mandatory-time-dimension field is completed; if the entity type and the field type mapping deviate in the entity extraction result, such as the entity only maps "department ID", but the RAG knowledge shows that "department query needs to contain ID and name fields at the same time to support chart display", the analysis subject field is corrected to department ID+department name; based on the entity-field mapping relationship output by the RAG, the field type is associated with the actual field name in the model, and finally a structured list is generated to clearly indicate the "field name, field type, and whether it is mandatory" corresponding to each core field type; from the output of the model selection link, the field list of the selected model is extracted, including "field name, field type, and field business meaning", forming a model field list; the core field requirement list of the user question and the model field list are compared item by item according to "field name+field type", and it is judged whether they are covered or not; if all field names corresponding to a "mandatory field type" do not appear in the model field list, it is determined that "mandatory field is missing"; if the field name exists but the field type does not match, it is determined that "the field type does not match"; if the field name corresponding to a "optional field type" does not appear in the model field list, it is determined that "optional field is missing"; if there are irrelevant fields in the model field list that are not in the core field requirement list, mark "field redundancy".

[0049] The model selection link of the present embodiment ensures that the selected model matches the user question scenario through scenario adaptability and historical effect verification.

[0050] In this embodiment, the output result of the question understanding link is checked in multiple dimensions in combination with the running state of the question understanding link, specifically: if the running state of the question understanding link is that the element recognition is completed, a historical similar question matching the user input question is determined based on semantic similarity, a standard understanding result of the historical similar question is extracted, and it is determined whether the understanding result output by the current question understanding link matches the standard understanding result in intention; based on a pre-defined question understanding core element list, it is checked whether the understanding result output by the current question understanding link covers all mandatory core elements; it is determined whether the understanding result output by the current question understanding link is in semantic conflict with the entity extraction result output by the entity extraction link, and whether the understanding result output by the current question understanding link matches the model field list of the selected model output by the model selection link.

[0051] This embodiment verifies the logical matching degree of the question understanding result and the original demand of the user, to avoid answering irrelevant questions.

[0052] As a specific implementation, the decision intelligent agent is called to disassemble the core query intention of the user input question, and then the query intention interpretation output by the question understanding is compared, if there is a logical deviation between the two, it is marked as “intention error”.

[0053] From the historical question library, a historical similar question with a semantic similarity TOPX with the current user question is matched, the standard understanding result of these similar questions is extracted, the intention matching degree of the current understanding result and the standard understanding result is calculated, and if the matching degree is lower than a pre-set threshold such as 85%, it is marked as “insufficient logical consistency”.

[0054] As a specific implementation, based on a pre-defined question understanding core element list, it is checked whether the understanding result covers all mandatory elements, to avoid deviation of downstream SQL generation due to missing elements. By checking one by one whether the understanding result output by the question understanding contains all the above mandatory elements, if any mandatory element is missing, it is marked as “element missing” and the specific element type missing is noted; if the mandatory element information is ambiguous, it is marked as “element information is not clear”. The pre-defined question understanding core element list includes query subject, core index, time dimension, calculation method and filtering condition, etc.

[0055] For example, the pre-defined question understanding core element list of the user input question includes query subject, core index, time dimension, calculation method and filtering condition, etc. As a specific implementation, the core keywords output by the question understanding output are compared with the core keywords output by the preceding entity extraction link. If the key information in the understanding result is not in the keyword list of entity extraction, or there is a semantic conflict with the keywords, mark "not matched with the entity extraction result". The query model of the question understanding output is compared with the selected model ID and the model field list output by the preceding model selection link. If the query model in the understanding result is inconsistent with the selected model ID, or the use field is not in the field list of the selected model, mark "not matched with the model selection result".

[0056] For example, the core keywords output by the entity extraction link contain "East China region", and if the filter condition of the question understanding result is "North China region", mark "not matched with the entity extraction result". The "profit table model V3" is selected by the model selection link, and "profit table model V3" contains the "revenue amount" field, but not the "profit rate" field. If the core indicator of the question understanding result is "profit rate", mark "not matched with the model selection result".

[0057] The question understanding link of the embodiment is verified by logical consistency and element integrity to ensure that the understanding result aligns with the user demand.

[0058] In the embodiment, the output result of the NL2SQL link is verified in multiple dimensions in combination with the running state of the NL2SQL link. Specifically, the SQL output by the NL2SQL link is analyzed for syntax parsing to verify the syntax rules, and the table name and field name of the SQL are verified in combination with the database metadata and the field list of the selected model. If the syntax parsing passes and the metadata query succeeds, the aggregation function, dimension field and measure field in the SQL are compared with the output result of the question understanding link to determine whether the business logic of the SQL aligns with the user demand output by the question understanding link. Based on semantic similarity, the historical similar questions matching the user input question are determined, the standard SQL of the historical similar questions is extracted, the similarity between the SQL output by the current NL2SQL link and the standard SQL is calculated, and the logical rationality of the current SQL is verified.

[0059] As a specific implementation, a SQL syntax parser is called to perform syntax analysis on the generated SQL to check whether there are problems such as keyword spelling errors, missing syntax structure, open parentheses, open quotes, etc. If there are, mark "SQL syntax error" and locate the error position.

[0060] Combine the database metadata and the field list of the selected model to check if the table name and field name in the SQL are real and accessible: if the table specified in the SQL does not exist in the database or the current user has no query permission, mark "table does not exist / no permission"; if the field used in the SQL is not in the field list of the selected model or the field name is misspelled, mark "field does not exist / misspelling"; check if the use of the field in the SQL is compatible with the field type, and if not, mark "field type use error".

[0061] As a specific implementation, compare the aggregation function in the SQL with the calculation method output by the problem understanding link: if the problem understanding requires summation, the SQL uses AVG, i.e., average, or if the problem understanding requires counting, the SQL uses SUM, i.e., summation, mark "aggregation function error".

[0062] Extract the filtering conditions of the WHERE clause in the SQL and perform text and semantic matching with the filtering conditions of the problem understanding link: if the SQL lacks key filtering conditions, mark "filtering condition missing"; if the SQL filtering conditions conflict with the requirements, mark "filtering condition error"; if the SQL has redundant filtering conditions and is not mentioned in the problem understanding notes, mark "filtering condition redundancy".

[0063] Check if the dimension field and the measure field of the SELECT clause in the SQL are consistent with the query dimension and the core indicator of the problem understanding link: if a key dimension is missing, mark "dimension missing"; if the measure field is inconsistent with the core indicator, mark "measure field error".

[0064] For example, the problem understanding link defines the core indicator = total revenue, the filtering condition = East China region + 2024Q3, and the query dimension = department, and if the generated SQL is SELECT department name, AVG(revenue amount) FROM bi.sale_2024 WHERE region = 'East China', AVG ≠ SUM, and 2024Q3 is missing, there are respectively marks "aggregation function error" and "filtering condition missing".

[0065] The NL2SQL link of the present embodiment corrects SQL syntax errors and aggregation function misuse in advance through syntax compliance and business logic verification, avoiding invalid results returned after executing the error SQL.

[0066] In the embodiment, the output result of the SQL execution link is checked in multiple dimensions in combination with the running state of the SQL execution link. Specifically, if the running state of the SQL execution link is an execution error, error key information is extracted from the error log in the SQL execution process, the error key information is matched with a preset SQL error type library, and the error root cause is determined according to the matching result. If the running state of the SQL execution link is execution success, the data dimension of the SQL statement query result is matched and checked with the query dimension output by the problem understanding link, and the numerical fields of the SQL statement query result are checked based on the field data range obtained by the model selection link.

[0067] As a specific implementation, error logs in the SQL execution process are captured, error key information is extracted, a preset SQL error type library is called, the collected error information is matched with the classification and root cause in the library, and the explicit error root cause is output according to the matching result.

[0068] The SQL error type library pre-stores a mapping relationship of error category-error scene-corresponding root cause-responsible link. For example, syntax error-table does not exist / field error / missing syntax-NL2SQL link generated SQL does not conform to database syntax specification-NL2SQL link; data error-empty result set / data type conversion failure-severe filtering condition / no corresponding data / field type error- SQL execution link.

[0069] As a specific implementation, the data dimension in the SQL statement query configuration information is compared with the query dimension output by the previous problem understanding link. If a key dimension is missing, it is marked as “dimension missing”. If there are redundant irrelevant dimensions and they are not explained in the query configuration information, it is marked as “dimension redundancy”. Based on the field data reasonable range provided by the previous model selection link, the numerical fields in the query result are checked. If the value exceeds the reasonable range ± 50%, it is marked as “data anomaly (value exceeds the standard)”. If there is logical contradictory data, it is marked as “data anomaly (logical contradiction)”. If the previous problem understanding link contains TopN sorting requirement, it is checked specifically. For example, if the requirement is TOP5 and 10 data are returned in the result, it is marked as “quantity error (exceeds TopN requirement)”. For example, if the requirement is “descending order”, the result is arranged in ascending order of revenue, and it is marked as “sorting error (opposite to the requirement)”. It is judged whether the query result can be directly used for front-end chart rendering to avoid data without actual display significance. If the result only contains the coding field and lacks the name field, the chart cannot display the label understandable by the user, and it is marked as “insufficient adaptability (lack of name field)”. If the result data format does not meet the front-end requirement, it is marked as “insufficient adaptability (format error)”.

[0070] For example, the problem understanding link requirement is "2024Q3 East China regional revenue TOP5 department (in descending order of revenue)", the query result returns "department ID, revenue amount" (without "department name"), and returns 8 data, sorted in ascending order of revenue, so mark "dimension missing (lack of department name)" "quantity error (8 ≠ 5)" "sorting error (ascending ≠ descending)" "insufficient adaptability (lack of name field)".

[0071] The SQL execution link in this embodiment passes through the front-end adaptability check, ensuring that the result contains a name field that users can understand, and the data format meets the chart rendering requirements, avoiding correct results that cannot be used.

[0072] Step 2: According to the check results of the output results of each link, generate a standardized problem identifier, and determine the corresponding error type and matching processing method based on the pre-set keyword library, and modify and optimize the corresponding process link based on the error type.

[0073] In this embodiment, the pre-set keyword library is a mapping association table of problem identifier-error type-processing method-priority, and the problem identifier corresponds one-to-one to the standardized problem identifier.

[0074] Based on the ChatBI full-process entity extraction, model selection, problem understanding, NL2SQL, execution SQL, and result check, the standardized problem identifier is generated following the link positioning and exception marking coding rules, ensuring that the identifier is unique, traceable, and directly related to the error root. Among them, the link positioning uses English abbreviations such as entity extraction = EE, model selection = MS, etc., the error type uses two-digit codes such as information missing = 01, matching error = 02, etc., and the subdivided scene uses Chinese keywords for supplementation.

[0075] In this embodiment, after a certain process link generates a check report, the link elements, error manifestation elements, and evidence support elements are extracted through keyword matching and semantic analysis, the extracted link elements are matched with the pre-set link-coding mapping table to generate the link positioning code of the identifier, the problem is classified into the N major error types pre-set in the scheme according to the error manifestation elements, and a two-digit code is assigned to form the error type code of the identifier, and the Chinese subdivided scene description is supplemented according to the specific details of the error manifestation elements to form the subdivided scene code of the identifier, ensuring that the identifier can directly reflect the specific scene of the problem. The rule is: subdivided scene = core error object + error behavior.

[0076] For example, the link code "EE" + error type code "01" + subdivided scene "Q3 keyword omission" → complete identifier "EE-01-Q3 keyword omission", corresponding to the entity extraction link not extracting the 'Q3' keyword.

[0077] In this embodiment, after the entity extraction link verification is completed, a standardized verification conclusion containing the verification state, exception label and exception details is generated, a standardized problem identification is generated according to the standardized verification conclusion, and then the decision intervention mode is determined: if it is a slight deviation such as RAG recall redundancy, the RAG screening intelligent agent is called to discard irrelevant knowledge; if it is a serious error such as entity extraction exception, manual intervention is triggered, the user is prompted to supplement key information or modify the entity extraction rule, and the verification result is recorded to the process optimization log for subsequent algorithm iteration of the entity extraction intelligent agent.

[0078] In this embodiment, after the model selection link verification is completed, a standardized verification conclusion containing the verification state, exception label and exception details is generated, a standardized problem identification is generated according to the standardized verification conclusion, and then the decision intervention mode is determined: if it is a slight deviation such as “insufficient scene adaptability but complete fields”, the RAG is called to supplement “model expansion business description” to retrigger the model selection intelligent agent to optimize selection; if it is a serious error such as “model mismatch” “core field missing”, manual intervention is triggered, the “error link + recommended model list” is pushed, and the model selection link is restarted after manual confirmation; at the same time, the verification result is recorded to the process optimization log for subsequent reverse optimization of the model selection strategy.

[0079] In this embodiment, after the problem understanding link verification is completed, a standardized verification conclusion containing the verification state, exception label and exception details is generated, a standardized problem identification is generated according to the standardized verification conclusion, and then the decision intervention mode is determined: if it is a slight deviation such as “element information is not clear”, the RAG is called to supplement business knowledge such as specifying the specific quarter of the time dimension to retrigger the problem understanding intelligent agent to correct the result; if it is a serious error such as “intention error” “core element missing” “serious mismatch with the previous link”, manual intervention is triggered, the “user original question + previous link output + current understanding result” is pushed, and the link is restarted after the understanding logic is manually corrected; at the same time, the verification result is recorded to the process optimization log for subsequent reverse optimization of the problem understanding intelligent agent algorithm.

[0080] In this embodiment, after the NL2SQL link verification is completed, a standardized verification conclusion containing the verification state, exception label and exception details is generated, a standardized problem identification is generated according to the standardized verification conclusion, and then the decision intervention mode is determined: if it is a slight deviation such as “filter condition redundancy” “SQL optimization space is insufficient” and the like, the SQL repair intelligent agent is called to regenerate the SQL; if it is a serious error such as “SQL syntax error” “aggregation function error” and the like, manual intervention is triggered, the “user demand + problem understanding output + current SQL + exception label” is pushed, and the link is restarted after the SQL is manually corrected; at the same time, the verification result is recorded to the process optimization log for subsequent reverse optimization of the NL2SQL intelligent agent.

[0081] In this embodiment, after the SQL segment verification is completed, a standardized verification conclusion containing the verification state, exception flag and exception details is generated. According to the standardized verification conclusion, a standardized problem identification is generated, and then the decision intervention mode is determined: if it is a slight deviation such as “dimension redundancy” or “data format error”, the data processing intelligent agent is called to correct the result; if it is a serious error such as “syntax error”, manual intervention is triggered: if the responsible link is NL2SQL, the error information is pushed to the NL2SQL link to regenerate SQL; if it is a data logic problem, the user is prompted to adjust the filtering condition or check the database data; at the same time, the verification result is recorded in the process optimization log for subsequent reverse optimization: if “empty result set” frequently occurs, the filtering condition generation rule of NL2SQL is optimized; if “data exception” frequently occurs, the reasonable range threshold of the model field is updated.

[0082] In this embodiment, if each link verification passes and the intervention repair is completed, the process normally proceeds to “result output”, and the final result is fed back to the user; if the manual intervention still cannot be repaired, the process is marked as failed, the user is fed back the exception and prompted for manual assistance, and the failure case is recorded for subsequent optimization.

[0083] The embodiment scheme takes full-link monitoring-accurate verification-layered intervention-closed-loop optimization as the core, covers the key links of user problem handling, takes into account automatic repair and manual backup, and guarantees system reliability.

[0084] The embodiment also includes: adjusting the current problem handling link based on log data, such as the RAG link of a certain type of problem frequently “recalling insufficient”, then the high-recall vector model is called to realize real-time optimization; optimizing the algorithm rules of each intelligent agent based on logs, for example, the entity extraction link frequently “misses the time dimension”, then the keyword recognition library is updated; the NL2SQL link frequently “misses the condition”, then the filtering condition generation rule in the prompt word template is optimized.

[0085] As an optional implementation, a larger size and better performance intelligent agent is used to complete the optimization of the process link, and the designed prompt word contains the input of each link, historical similar cases, error classification, and suggested repair actions, etc.

[0086] The prompt word example is as follows: You are a professional database query evaluation expert, and now you need to make a professional judgment on the output result of the multi-agent collaborative system. Please follow the following process to verify:

Output requirements

[0087] Verify that the extracted entity values ​​are accurate and that there are no omissions or errors.

[0088] If any omissions or errors are found, please record the specific missing fields or incorrect values.

[0089] Model selection validation: Evaluate whether the model selected by the system is suitable for handling the current user problem.

[0090] Review the model's performance on similar historical problems to confirm whether its selection was reasonable.

[0091] If the model is not appropriate, please explain the reason and recommend a more suitable model.

[0092] SQL generation verification: Check if the generated SQL statement is syntactically correct and free of syntax errors.

[0093] Verify that the WHERE clause in the SQL statement accurately reflects the filtering conditions in the user's question.

[0094] Confirm that the SQL statement uses the correct aggregation functions such as SUM, AVG, etc. and the sorting method.

[0095] If the SQL statement has logical errors or does not meet the user's intentions, please record the error type and location in detail.

[0096] Query result verification: Execute the generated SQL statement in the database to obtain the query result.

[0097] Match the query result with the user's expected result and check if the returned data dimensions are complete.

[0098] Verify that the data in numerical fields is within a reasonable range and there are no abnormal values.

[0099] If the query result is incorrect or there are exceptions, analyze the reasons and record them.

[0100] Overall logic verification: Evaluate whether the entire NL2SQL conversion process forms a closed loop, i.e. whether the entire process from user question to SQL statement to query result is logically smooth.

[0101] Check if there are errors or omissions in the information transmission process, such as inaccurate entity extraction leading to SQL statement errors, etc.

[0102] If there are problems in the overall logic, please point out the specific steps and error types.

[0103]

Notes

[0104] If the query result is correct but the process has hidden problems, it is still marked as an error.

[0105] For complex queries, it is recommended to verify the implementation of each filtering condition in steps.

[0106] Combine the information from historical problem inputs to verify the overall result.

[0107] } Example two The embodiment aims to provide a conversational business intelligence platform process optimization system, comprising: A checking module configured to: acquire output results of each link in the conversational business intelligence platform process, and sequentially perform multi-dimensional checking on the output results of each link in the order of process execution to determine whether the output results of the process links meet the output requirements; wherein the output requirements include adaptation to the original requirement of the user input question, compliance with the business logic of the process link itself, and adaptation to the use requirement of the downstream process link; An optimization module configured to: generate a standardized question identifier according to the checking results of the output results of each link, determine the corresponding error type and matching processing mode in combination with a preset keyword library with the standardized question identifier as an index, and realize correction and optimization of the corresponding process link In more embodiments, there are also provided: An electronic device comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the method described in embodiment one. For brevity, this will not be described here.

[0108] It should be understood that in the embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0109] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0110] A computer readable storage medium for storing computer instructions, which, when executed by a processor, complete the method described in embodiment one.

[0111] The method in embodiment one can be directly embodied as hardware processor execution completion, or executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0112] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the present embodiment can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0113] Although the specific embodiments of the present application are described above in combination with the drawings, it is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A method for optimizing processes in a conversational business intelligence platform, characterized in that, include: The system acquires the output results of each stage in the conversational business intelligence platform process and performs multi-dimensional verification on the output results of each stage in the order of process execution to determine whether the output results of each stage meet the output requirements. The output requirements include adapting to the original requirements of the user input question, conforming to the business logic of the process stage itself, and adapting to the usage requirements of downstream process stages. Based on the verification results of the output of each step, standardized problem identifiers are generated. Using the standardized problem identifiers as an index and combined with a pre-set keyword library, the corresponding error types and matching processing methods are determined, thereby realizing the correction and optimization of the corresponding process steps.

2. The process optimization method for a conversational business intelligence platform as described in claim 1, characterized in that, Obtain the output results of each stage in the conversational business intelligence platform process, and perform multi-dimensional verification on the output results of each stage in the order of process execution to determine whether the output results of each stage meet the output requirements. Specifically: Obtain the running status and output results of the entity extraction, model selection, problem understanding, NL2SQL, and SQL execution stages in the conversational business intelligence platform process; Based on the operational status of the corresponding process steps, the output results of the entity extraction step, model selection step, problem understanding step, NL2SQL step, and SQL execution step are verified in multiple dimensions according to the process execution order to determine whether the output results of the process steps meet the output requirements.

3. The process optimization method for a conversational business intelligence platform as described in claim 2, characterized in that, Based on the operational status of the entity extraction process, the output results of the entity extraction process are verified from multiple dimensions, specifically: If the entity extraction process is running in the state of keyword matching success, then the character similarity matching and core element comparison are used to determine whether the entity extraction result output by the current entity extraction process is semantically aligned with the user input question. Determine whether the candidate models for enhanced retrieval service based on entity recall belong to the model type that is strongly associated with the extracted entities, and determine whether the business knowledge recalled by enhanced retrieval service is semantically consistent with the extracted entities; Based on semantic similarity, historical similar questions matching the user input question are determined. The difference between the standard entities of the historical similar questions and the extracted entities of the user input question is calculated to verify the rationality of the entity extraction results.

4. The process optimization method for a conversational business intelligence platform as described in claim 2, characterized in that, Based on the operational status of the model selection process, the output results of the model selection process are validated from multiple dimensions, specifically as follows: If the model selection process is in the "model matching successful" state, determine whether the topic tags of the business description of the selected model output by the current model selection process are consistent with the topic tags of the user input question; wherein, the topic tags are used to indicate the business scenario; Based on the matching of user input questions and corresponding topic tags, historical similar questions are obtained, and the model selection results of the current user input question are verified using the model selection results of historical similar questions. Based on the entity extraction results from the entity extraction stage and the business knowledge of the enhanced retrieval service, the core field types required by the user input question are analyzed, and the field list of the selected model is verified to cover the core field types required by the user input question.

5. The process optimization method for a conversational business intelligence platform as described in claim 2, characterized in that, Based on the operational status of the problem understanding phase, the output results of the problem understanding phase are verified from multiple dimensions, specifically: If the problem understanding process is in the state of element recognition completion, then based on semantic similarity, determine the historical similar problems that match the user input problem, extract the standard understanding results of the historical similar problems, and determine whether the understanding result output by the current problem understanding process matches the standard understanding result. Based on a predefined list of core elements for problem understanding, verify whether the understanding results output in the current problem understanding stage cover all the required core elements; Determine whether the understanding result output by the current problem understanding stage conflicts semantically with the entity extraction result output by the entity extraction stage, and whether it matches the list of model fields of the selected model output by the model selection stage.

6. The process optimization method for a conversational business intelligence platform as described in claim 2, characterized in that, Based on the operational status of the NL2SQL step, the output results of the NL2SQL step are validated from multiple dimensions, specifically as follows: The SQL output from the NL2SQL stage is parsed and analyzed to verify the syntax rules. In addition, the table names and field names of the SQL are verified by combining the database metadata and the field list of the selected model. If the syntax parsing passes and the metadata query is successful, the aggregate functions, dimension fields, and metric fields in the SQL will be compared with the output results of the problem understanding stage to determine whether the business logic of the SQL is aligned with the user requirements output by the problem understanding stage. Based on semantic similarity, historical similar questions matching the user input question are determined. The standard SQL of the historical similar questions is extracted, the similarity between the SQL output of the current NL2SQL step and the standard SQL is calculated, and the logical rationality of the current SQL is verified.

7. The process optimization method for a conversational business intelligence platform as described in claim 2, characterized in that, Based on the execution status of the SQL execution phase, the output results of the SQL execution phase are validated from multiple dimensions, specifically: If the execution status of the SQL execution stage is "execution error", then extract the key error information from the error log during the SQL execution process, match the key error information with the preset SQL error type library, and determine the root cause of the error based on the matching result; If the execution status of the SQL step is successful, then the data dimensions of the SQL query results are matched and verified with the query dimensions output by the problem understanding step, and the numeric fields of the SQL query results are verified based on the field data range obtained by the model selection step.

8. The process optimization method for a conversational business intelligence platform as described in claim 1, characterized in that, The pre-set keyword library is a mapping association table of problem identifier, error type, processing method, and priority, and the problem identifier corresponds one-to-one with the standardized problem identifier.

9. A conversational business intelligence platform process optimization system, characterized in that, include: The verification module is configured to: obtain the output results of each step in the conversational business intelligence platform process, and perform multi-dimensional verification on the output results of each step in the order of process execution to determine whether the output results of the process steps meet the output requirements; wherein, the output requirements include adapting to the original requirements of the user input question, conforming to the business logic of the process step itself, and adapting to the usage requirements of downstream process steps. The optimization module is configured to generate standardized problem identifiers based on the verification results of the outputs of each step, and use the standardized problem identifiers as an index in conjunction with a pre-set keyword library to determine the corresponding error types and matching processing methods, thereby realizing the correction and optimization of the corresponding process steps.

10. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-8.

Images