Intelligent dialogue analysis method, system and device based on multi-stage semantic matching

Abstract

The application provides a kind of intelligent dialogue analysis method, system and equipment based on multi-stage semantic matching, the method obtains user instruction;Intention extraction is carried out on user instruction using large model, according to the target intention obtained, recall multiple candidate sets, and filter the highest scoring candidate set as the best candidate set;Call semantic model to the natural language entity in the best candidate set Semantic analysis is carried out, and a semantic query statement is generated, the semantic model is used to form an index for the mapping relationship between the natural language entity and the physical data model in the knowledge base;The semantic query statement is corrected and converted, and the executable SQL statement is determined;According to the executable SQL statement, the data source is queried, the query result is determined and displayed;Through semantic analysis, semantic ambiguity is reduced, and analysis reliability is improved;The semantic query statement is corrected and converted, the statement error rate is reduced, and the query execution stability is improved.

Description

Technical Field

[0001] This application relates to the field of large-scale models or intelligent dialogue, and in particular to an intelligent dialogue analysis method, system and device based on multi-stage semantic matching. Background Technology

[0002] In the process of enterprise digital transformation, data analytics has become a crucial support for business decision-making. On the one hand, traditional data analytics systems require users to master SQL (Structured Query Language), posing a technical barrier and limiting its use by ordinary users. On the other hand, while some Natural Language Query (NLQ) systems or conversational BI tools have emerged in recent years, they still suffer from the following technical shortcomings in practical applications, especially in complex enterprise-level environments with multiple datasets:

[0003] First, because simple keyword matching or single-dimensional similarity determines the dataset, it is difficult to cope with the noise of synonyms, business terms and deep semantic understanding of query intent, resulting in insufficient matching accuracy of the dataset. Second, the process of mapping from natural language to physical SQL lacks abstraction of the business semantic layer and effective intermediate representation, which can easily lead to semantic ambiguity or illegal query statements. At the same time, the lack of a verification and correction mechanism for the generated SQL results in poor robustness of semantic parsing and SQL generation. Third, the system architecture is rigid, making it difficult to integrate the latest AI capabilities (such as more powerful LLM or embedded models) and adapt to the diverse data infrastructure already available to enterprises. Therefore, there is an urgent need for an intelligent dialogue analysis solution that can accurately understand user intent, has robust query capabilities, and is highly scalable. Summary of the Invention

[0004] This application provides an intelligent dialogue analysis method, system, and device based on multi-stage semantic matching, to solve the problems of insufficient dataset matching accuracy, semantic parsing robustness, and system scalability in existing intelligent dialogue analysis based on multi-stage semantic matching.

[0005] In a first aspect, this application provides an intelligent dialogue analysis method based on multi-stage semantic matching, comprising: acquiring user instructions, wherein the user instructions are represented as textual instructions; extracting intent from the user instructions using a large model, recalling multiple candidate sets based on the obtained target intent, and selecting the candidate set with the highest score as the best candidate set; calling a semantic model to perform semantic parsing on the natural language entities in the best candidate set, generating a semantic query statement, wherein a pre-built semantic model is used to index the mapping relationship between natural language entities and physical data models in the knowledge base; performing semantic correction on the semantic query statement, and transforming the corrected semantic query statement to determine an executable SQL statement; querying the data source based on the executable SQL statement, determining the query results, and displaying them.

[0006] In some possible embodiments of the first aspect, the intent extraction of user instructions using a large model includes: using a large model to clean the user instructions, the data cleaning including denoising and polite language filtering, the denoising including data deduplication, missing value imputation and outlier filtering; using a large model to extract the intent of the cleaned user instructions, and calling the corresponding semantic model to standardize and map the business terms in the intent based on the extracted intent to determine the target intent.

[0007] In some possible embodiments of the first aspect, recalling multiple candidate sets based on the obtained target intent and selecting the candidate set with the highest score as the best candidate set includes: converting the target intent into a query vector, calculating the similarity between the query vector and the vector dataset represented by each dataset, selecting a preset number of datasets with the highest similarity as multiple candidate sets; using a large model to perform weighted calculation on each candidate set, and determining the candidate set with the highest score as the best candidate set, wherein the weight of each candidate set is positively correlated with the fields, names and descriptions in the candidate set.

[0008] In some possible embodiments of the first aspect, the semantic model is constructed in the following ways: configuring a semantic model for a large model based on business parameters, including business topics, metrics, and dimensions; using the semantic model to extract the mapping relationship between the dataset, metrics, and dimensions, constructing an index and storing it in a knowledge base; and constructing a pattern mapper to enable natural language entities in user instructions to perform term matching and semantic association in the knowledge base.

[0009] In some possible embodiments of the first aspect, invoking a semantic model to perform semantic parsing on natural language entities in the best candidate set and generating a semantic query statement includes: invoking a semantic model to find natural language entities in the best candidate set, matching the corresponding context in the physical data model in the knowledge base; and performing semantic parsing based on the best candidate set and the corresponding context to determine the semantic query statement.

[0010] In some possible embodiments of the first aspect, semantic correction is performed on the semantic query statement, and the corrected semantic query statement is converted to determine an executable SQL statement, including: performing semantic correction on the semantic query statement to determine the corrected query statement, wherein the semantic correction includes at least one of integrity check, logical consistency check, syntax legality check, and security check; converting the corrected query statement into a physical query statement that conforms to the dialect of the target database, and determining it as an executable SQL statement.

[0011] In some possible embodiments of the first aspect, before extracting intent from user instructions using a large model, the method further includes: structurally encapsulating user instructions, semantic query statements, executable SQL, and query results to form a historical dialogue trajectory and storing it; in a new dialogue, when a natural language query with semantic similarity to the historical dialogue trajectory is detected, recalling at least one historical interaction record from the historical dialogue trajectory that is identical to the current query domain, intent, metric, or dimension as a sample example; and concatenating the sample example with the current user's natural language query to form a prompt word.

[0012] In some possible embodiments of the first aspect, the method further includes at least one of the following: if the large model intelligent reordering is abnormal or the confidence of the reordering result is still lower than a preset threshold, then the candidate set ranked first in recall is used as the best candidate set; if the large model semantic parsing times out, malfunctions, or the parsing result is invalid, then a predefined query template is matched from a preset template library, and parameters are automatically filled to generate a semantic query statement.

[0013] In a second aspect, this application provides an intelligent dialogue analysis system based on multi-stage semantic matching, comprising: an acquisition module for acquiring user instructions, wherein the user instructions are represented as textual instructions; a semantic processing module for extracting intent from user instructions using a large model, recalling multiple candidate sets based on the obtained target intent, and selecting the candidate set with the highest score as the best candidate set; a semantic matching module for calling a semantic model to perform semantic parsing on natural language entities in the best candidate set, generating semantic query statements, wherein a pre-built semantic model is used to index the mapping relationship between natural language entities and physical data models in a knowledge base; a semantic correction module for semantically correcting the semantic query statements, and transforming the corrected semantic query statements to determine executable SQL statements; and a query module for querying the data source based on the executable SQL statements, determining the query results, and displaying them.

[0014] In a third aspect, this application also provides an electronic device, including a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the intelligent dialogue analysis system based on multi-stage semantic matching as described above.

[0015] The beneficial effects of this application are as follows: A large model is used to extract user intent from commands. Based on the obtained target intent, multiple candidate sets are recalled, and the candidate set with the highest score is selected as the best candidate set. Through large-model intent extraction and precise candidate set selection, the accuracy of intent understanding and candidate matching precision are improved. A semantic model is invoked to perform semantic parsing on natural language entities in the best candidate set, generating semantic query statements. The pre-built semantic model is used to index the mapping relationship between natural language entities and physical data models in the knowledge base. Establishing an entity-physical data model mapping index through the semantic model reduces semantic ambiguity and improves parsing reliability. Semantic correction is performed on the semantic query statements, and the corrected semantic query statements are transformed to determine executable SQL statements. Through semantic correction and SQL transformation, the statement error rate is reduced, and the query execution stability is improved. The data source is queried based on the executable SQL statements, and the query results are determined and displayed. Through standardized query and result display, query efficiency and accuracy are improved, and the cost of manually writing SQL statements is reduced. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram: Figure 1 A flowchart of an intelligent dialogue analysis method based on multi-stage semantic matching provided in an embodiment of this application; Figure 2 A structural block diagram of an intelligent dialogue analysis system based on multi-stage semantic matching provided in an embodiment of this application; Figure 3 A structural block diagram of an intelligent dialogue analysis system based on multi-stage semantic matching provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0021] In related technologies, natural language to database query technology suffers from problems such as inaccurate understanding of user intent, low efficiency in candidate set filtering, fuzzy mapping between natural language entities and physical data models, large conversion errors in semantic query statements, and insufficient reliability of SQL statement execution. These issues result in a large deviation between query results and actual user needs, and the query efficiency and accuracy are difficult to meet the requirements of industrial applications.

[0022] To resolve the above issues, please refer to [link / reference]. Figure 1 The above is a flowchart illustrating an intelligent dialogue analysis method based on multi-stage semantic matching provided in an embodiment of this application, including: Step S110: Obtain user instructions, which are represented as text-based instructions; For example, a query request input by the user is received through a human-computer interaction interface. The query request can be voice input, gesture input, lip reading input, text input, or video input. For example, textual instructions can be obtained by means of speech recognition to text, OCR (Optical Character Recognition) recognition to text, or unified multimodal instruction to text. Regardless of whether the query request is text input, the content of the query request needs to be recognized and converted into a textual instruction representation in order to obtain the user instruction.

[0023] Step S120: Use a large model to extract the intent of the user's command, recall multiple candidate sets based on the obtained target intent, and select the candidate set with the highest score as the best candidate set. For example, the large model includes, but is not limited to, Doubao, Qianwen, Yuanbao, etc. Textual instructions are input into the large model that has been fine-tuned by domain data, and the large model outputs the target intent; based on the target intent, a vector retrieval with keyword matching is performed in the candidate library to recall multiple candidate sets; a weighted score is adopted using intent matching degree, entity coverage, and business relevance, and the candidate set with the highest score is selected as the best candidate set.

[0024] Step S130: Call the semantic model to perform semantic parsing on the natural language entities in the best candidate set and generate a semantic query statement. The pre-built semantic model is used to form an index of the mapping relationship between natural language entities and physical data models in the knowledge base. For example, a mapping index between natural language entities and physical data models is constructed. Through entity alignment and semantic structuring, ambiguity in natural language can be eliminated. Specifically, natural language entities in the best candidate set are identified and labeled; through a pre-built mapping index of semantic models, entities are matched to the tables, fields, and relationships of the corresponding physical data models in the knowledge base; and standardized semantic query statements are generated based on the mapping relationships.

[0025] Step S140: Perform semantic correction on the semantic query statement, and transform the corrected semantic query statement to determine the executable SQL statement; For example, deviations in semantic query statements are corrected through rule validation and semantic constraints, avoiding invalid or erroneous queries. Template mapping and syntax tree transformation are used to convert standardized semantic structures into SQL statements that can be directly executed by the database. For instance, based on database syntax specifications and business logic rules, logical validation, entity integrity validation, and association rationality validation are performed on semantic query statements to complete semantic correction. The corrected semantic query statements are then structurally transformed according to SQL syntax templates to generate syntactically valid and logically correct executable SQL statements.

[0026] Step S150: Query the data source based on the executable SQL statement, determine the query results, and display them.

[0027] For example, an executable SQL statement is sent to the corresponding data source to execute the query and obtain the original query results; the original results are then processed by formatting, sorting, filtering, etc., and the query results are displayed in a visual interface or a standardized format.

[0028] By employing the above methods, a large-scale model is used to extract user intent from commands. Based on the obtained target intent, multiple candidate sets are recalled, and the candidate set with the highest score is selected as the best candidate set. Through large-scale model intent extraction and precise candidate set selection, the accuracy of intent understanding and candidate matching precision are improved. A semantic model is invoked to perform semantic parsing on the natural language entities in the best candidate set, generating semantic query statements. The pre-built semantic model is used to index the mapping relationship between natural language entities and physical data models in the knowledge base. By establishing an entity-physical data model mapping index through the semantic model, semantic ambiguity is reduced and parsing reliability is improved. The semantic query statements are semantically corrected and transformed to determine executable SQL statements. Through semantic correction and SQL transformation, the statement error rate is reduced and the query execution stability is improved. The data source is queried based on the executable SQL statements, the query results are determined and displayed. Through standardized query and result display, end-to-end automation of natural language to database query is achieved, improving overall query efficiency and accuracy, and reducing the cost of manually writing SQL statements.

[0029] Optionally, in some embodiments, intent extraction of user commands is performed using a large model, including: The large model is used to clean user commands. Data cleaning includes noise reduction and polite language filtering. Noise reduction includes data deduplication, missing value imputation and outlier filtering. The intent of user commands after data cleaning is extracted using a large model. Based on the extracted intent, the corresponding semantic model is invoked to standardize and map the business terms in the intent to determine the target intent.

[0030] For example, user commands are input into a large model, which then performs data cleaning operations on them. Noise removal includes: eliminating duplicate text fragments to remove duplicates; filling in missing values ​​by adding content to semantically incomplete sections; and filtering outliers by removing garbled characters, special characters, and irrelevant information without actual business meaning. Politeness filtering involves identifying and removing greetings, formalities, and other non-business-related polite expressions from user commands using the large model, while retaining core business semantic content. By identifying and processing interference, redundancy, missing information, and anomalies in user commands using the large model, invalid content is removed and semantically missing parts are filled in, reducing the interference of non-business information on the intent extraction process and improving the quality of input data.

[0031] For example, a large-scale model is used to convert unstructured user commands into structured intents. A mapping relationship between synonymous business terms and standard terms is established based on a semantic model, eliminating semantic ambiguity caused by the diversity of terminology. For instance, a user command to complete data cleaning is input into the large-scale model, which parses it based on contextual semantics and business scenario features to extract the intent. According to the business domain to which the extracted intent belongs, the corresponding semantic model for that business dimension is invoked to map business terms with different expressions but the same meaning within the intent to a pre-defined standard terminology system, completing the standardization conversion of terms. The target intent is then determined based on the standardized terms and the parsed intent.

[0032] Through the above methods, the large model performs data cleaning operations such as noise reduction and polite language filtering on user commands, eliminating redundant and interfering information; the intent extraction is combined with the semantic model to standardize the mapping of business terms, unify the terminology and eliminate semantic ambiguity, thereby improving the accuracy and standardization of intent recognition.

[0033] Optionally, in some embodiments, multiple candidate sets are recalled based on the obtained target intent, and the candidate set with the highest score is selected as the best candidate set, including: The target intent is converted into a query vector, the similarity between the query vector and the vector dataset represented by each dataset is calculated, and the top few datasets with the highest similarity are selected as multiple candidate sets. The large model is used to perform weighted calculations on each candidate set, and the candidate set with the highest score is determined as the best candidate set. The weight of each candidate set is positively correlated with the fields, names and descriptions in the candidate set.

[0034] For example, the target intent is semantically encoded, transforming the textual semantics of the target intent into a query vector of fixed dimensions; all datasets to be matched are pre-encoded to generate a vector dataset corresponding to each dataset, wherein the vector dataset corresponds one-to-one with the core semantic features of the dataset; a preset similarity calculation algorithm is used, such as cosine similarity, Euclidean distance, and Pearson correlation coefficient, to calculate the similarity between the query vector and each vector dataset, obtaining the similarity value between each dataset and the target intent; all similarity values ​​are sorted in descending order, and a preset number of datasets with the highest ranking are selected as multiple candidate sets, the preset number of which can be configured according to the actual business scenario requirements.

[0035] The initial candidate sets are input into a pre-defined large model, and the completeness of fields, semantic matching degree of names, detail of descriptions, and semantic coherence of each candidate set are analyzed. Based on the analysis results, corresponding weights are assigned to each candidate set. The more complete the fields of a candidate set, the higher the semantic matching degree between the name and the target intent, and the more detailed the description, the higher its weight value. That is, the weight is positively correlated with the core features of the fields, names, and descriptions in the candidate set. According to the pre-defined weighted calculation rules, combined with the weights and similarity values ​​of each candidate set, a comprehensive weighted score is calculated for each candidate set. The scores of all candidate sets are sorted in descending order, and the candidate set with the highest score is selected as the best candidate set.

[0036] By using the above method, the target intent is converted into a query vector and the similarity with the vectors of each dataset is calculated. The candidate set is then filtered by ranking the similarity, quantifying the semantic relevance between the target intent and the dataset, eliminating irrelevant datasets, and improving the accuracy of the initial candidate set screening. A large model is used to combine the candidate set's own fields, names, and descriptive features for weighted scoring. The weights are positively correlated with the core features, accurately distinguishing the fit quality of each candidate set and improving the accuracy of the best candidate set screening.

[0037] Optionally, in some embodiments, the semantic model is constructed in the following ways: Configure semantic models for large models based on business parameters, including business themes, metrics, and dimensions. Semantic models are used to extract the mapping relationships between datasets, metrics, and dimensions, and an index is built and stored in a knowledge base. Construct a pattern mapper to enable term matching and semantic association of natural language entities in user commands within a knowledge base.

[0038] For example, business parameters from actual business scenarios are collected to determine the business themes, indicators, and dimensions included in the business parameters. The business themes are the scope of business applications, the indicators are the core quantitative or qualitative parameters of the business, and the dimensions are the basis for indicator breakdown and statistics. The collected business parameters are standardized by removing redundant information and unifying the parameter expression format to form a standardized set of business parameters. Based on the standardized set of business parameters, the preset large model is fine-tuned for business scenarios, and a semantic model adapted to the current business scenario is configured. This enables the semantic model to have the semantic understanding and extraction capabilities corresponding to the business themes, indicators, and dimensions, ensuring that the semantic model is highly matched with business requirements. All datasets, metrics, and dimensions from the business scenario are input into the semantic model to parse the inherent relationships between them. For example, the correspondence between datasets and metrics, the attribution relationship between metrics and dimensions, and the indirect relationship between datasets and dimensions are extracted to form a standardized mapping relationship set. The mapping relationship set is deduplicated and validated to remove invalid and erroneous mapping relationships, ensuring the accuracy and completeness of the mapping relationships. An index is built based on the validated mapping relationships. The index is classified according to business topics and associates the mapping information of datasets, metrics, and dimensions. The completed index is stored in the knowledge base according to preset storage rules for easy retrieval and retrieval.

[0039] Based on the indexes and mapping relationships stored in the knowledge base, a pattern mapper is constructed. Through a pre-set business terminology dictionary, natural language entities in user commands are identified and extracted. The extracted natural language entities are accurately matched with business terms (including datasets, metrics, and dimensions) in the knowledge base to achieve the correspondence between natural language entities and standard business terms. Based on the mapping relationships and indexes in the knowledge base, the matched standard terms are associated with the corresponding datasets, metrics, and dimensions to determine the business semantics and associated information corresponding to the natural language entities, thereby completing the semantic parsing and association of natural language entities in user commands.

[0040] By employing the above methods, a semantic model is configured for the large model based on business parameters, enabling the semantic model to adapt to specific business scenarios. This improves the targeting and accuracy of semantic extraction and eliminates the business adaptation bias of general models. The semantic model is used to extract the mapping relationship among the three entities and build an index to store it in the knowledge base, achieving standardized and structured management of the mapping relationship, reducing the difficulty of retrieving the mapping relationship, and improving data reusability. A pattern mapper is built to achieve term matching and semantic association of natural language entities, eliminating expression differences and clarifying business semantics. At the same time, it improves the accuracy of semantic parsing of user commands.

[0041] Optionally, in some embodiments, a semantic model is invoked to perform semantic parsing on natural language entities in the best candidate set to generate a semantic query statement, including: The semantic model is invoked to find natural language entities in the best candidate set, and the physical data model in the knowledge base is matched with the corresponding context. Semantic parsing is performed based on the best candidate set and its corresponding context to determine the semantic query statement.

[0042] For example, the best candidate set obtained from the screening is input into the semantic model. The semantic model is used to accurately identify and extract natural language entities in the best candidate set to determine the semantic and business attributes of the natural language entities. Based on the extracted natural language entities, the physical data model in the preset knowledge base is retrieved. Through the semantic association capability of the semantic model, the physical data model context corresponding to the natural language entity is matched. The context includes the table structure, field associations, data constraints and business logic relationships related to the entity in the physical data model, ensuring that the matched context is semantically consistent with and closely related to the natural language entity.

[0043] The semantic model combines the best candidate set with information such as tables, fields, and relationships from the physical data model context to transform the unstructured candidate set semantics and structured context information into standardized semantic expressions. Based on the standardized semantic expressions, semantic query statements that conform to business logic and are semantically complete are generated, ensuring that the semantic query statements can accurately map the natural language entities of the best candidate set and are highly adapted to the physical data model context.

[0044] By employing the above method, the semantic model is invoked to find natural language entities in the best candidate set and match them with the context corresponding to the physical data model in the knowledge base. This eliminates semantic ambiguity of natural language entities, clarifies the relationship between entities and data models, and improves the accuracy of entity matching. Semantic query statements are generated by jointly performing semantic parsing based on the best candidate set and context, achieving precise alignment between query intent and physical data model. This avoids the one-sidedness of semantic parsing and improves the completeness and adaptability of semantic query statements.

[0045] Optionally, in some embodiments, the semantic query statement is semantically corrected, and the corrected semantic query statement is transformed to determine an executable SQL statement, including: Semantic correction is performed on the semantic query statement to determine the corrected query statement. Semantic correction includes at least one of the following: integrity check, logical consistency check, syntax legality check, and security check. The corrected query statement is converted into a physical query statement that conforms to the dialect of the target database and is then identified as an executable SQL statement.

[0046] For example, a semantic query statement is obtained, and at least one semantic correction operation is performed on the semantic query statement; the integrity check is used to verify whether the semantic query statement contains the core elements required for the query, and automatically supplements or prompts for improvement when the core elements are missing; the logical consistency check is used to verify whether the logic of conditions, relationships, etc. in the statement is self-consistent, so as to eliminate logical contradictions; the syntax legality check is used to verify whether the statement conforms to the semantic query specification and corrects syntax errors; the security check is used to verify whether the statement contains risk items such as malicious queries and unauthorized access, so as to eliminate risky content; the corrected semantic query statement is reviewed, and after confirming that there are no omissions, it is determined as the corrected query statement.

[0047] Obtain the type of the target database and its corresponding dialect specifications, and establish mapping rules between the corrected query statement and the target database dialect. The mapping rules include the correspondence between semantic expressions and database dialect grammar, keywords, and functions. Based on the mapping rules, perform structured transformation on the corrected query statement, converting the semantic-level query logic into a physical query statement that conforms to the target database dialect requirements. For example, if the target database dialect requirements meet syntax verification and compatibility testing, and it is confirmed that it can be recognized and executed by the target database, it is determined as an executable SQL statement.

[0048] By employing the above methods, semantic correction of the semantic query statement is performed in multiple dimensions, eliminating defects in the statement's completeness, logic, syntax, and security, thereby improving the effectiveness and compliance of the query statement and reducing query failures, result deviations, and data security risks. The corrected query statement is then converted into an executable SQL statement that conforms to the dialect of the target database, eliminating dialect adaptation barriers and allowing direct execution of the query statement, thus reducing manual conversion costs and errors.

[0049] Optionally, in some embodiments, before extracting intent from user commands using a large model, the method further includes: User commands, semantic query statements, executable SQL, and query results are encapsulated in a structured manner to form a historical dialogue track, which is then stored. In a new dialogue, when a natural language query with semantic similarity to the historical dialogue trajectory is detected, at least one historical interaction record that is identical to the current query domain, intent, metric, or dimension is recalled from the historical dialogue trajectory as a sample example. The sample example is concatenated with the current user's natural language query to form the prompt word.

[0050] For example, all interactive data generated during the dialogue is collected, including the user's original natural language commands, generated semantic query statements, converted executable SQL statements, and the final query results; the data is encapsulated according to a preset structured format, the relationship between each data item is clarified, and information such as dialogue timestamps, query domains, and core intents are marked to form a standardized historical dialogue trajectory; the encapsulated historical dialogue trajectory is stored in a preset database or knowledge base according to preset storage rules.

[0051] During a new dialogue, the system receives natural language queries input by the user in real time, calls a semantic model to perform semantic parsing on the query, and extracts key features such as the domain, intent, metrics, and dimensions of the current query. Based on the extracted key features, the system retrieves stored historical dialogue trajectories and monitors whether there are any historical dialogue trajectories that are semantically similar to the current query through semantic similarity calculation. When a semantically similar historical dialogue trajectory is detected, the system selects historical interaction records that are identical to at least one of the domain, intent, metrics, and dimensions of the current query. The selected records are then deduplicated and normalized to determine them as sample examples.

[0052] The interaction information in the sample examples is extracted, including but not limited to historical user commands, core intents, and key fragments of corresponding semantic query statements, and redundant information is removed. According to the preset splicing rules, the interaction information of the sample examples is sequentially spliced ​​with the natural language query input by the current user, ensuring that the spliced ​​content is semantically coherent, logically clear, and can fully present the current query requirements and reference examples. The spliced ​​content is grammatically regulated and semantically validated. After confirming that it is unambiguous and free of redundancy, it is determined as a prompt word.

[0053] By using the above methods, various types of interactive data are structured, encapsulated, and stored to form historical dialogue trajectories, enabling systematic management and reuse of historical interaction information. Semantically similar queries are monitored, and relevant historical interaction records are recalled as sample examples to establish a connection between the current query and historical experience, thereby improving query association efficiency. Sample examples are concatenated with the current query to form prompt words, enriching the semantic information of the prompt words. This can help large models quickly understand query requests, improving the accuracy and efficiency of query processing.

[0054] Optionally, in some embodiments, the method further includes at least one of the following: If the large model intelligent rearrangement is abnormal or the confidence of the rearrangement result is still lower than the preset threshold, the candidate set ranked first in recall will be used as the best candidate set. If the semantic parsing of the large model times out, encounters an error, or the parsing result is invalid, a predefined query template will be matched from the preset template library, and parameters will be automatically filled in to generate a semantic query statement.

[0055] For example, after the candidate set intelligent rearrangement operation is performed on the large model, the system monitors in real time whether any abnormalities occur during the rearrangement process. Abnormalities include the rearrangement process being interrupted, the rearrangement algorithm failing to execute, and the rearrangement result being empty. At the same time, the confidence level of the rearrangement result is calculated and compared with a preset threshold. If a rearrangement abnormality is detected, or the confidence level of the rearrangement result is lower than the preset threshold, a degradation mechanism is triggered. The large model's rearrangement result is abandoned, and the ranking result of the candidate set recall stage is directly called. The candidate set ranked first in the recall is extracted and determined as the best candidate set, ensuring the normal acquisition of the best candidate set and guaranteeing the continuous progress of the query process.

[0056] For example, during the semantic parsing operation of a large model, the parsing status is monitored in real time to determine whether timeouts (e.g., exceeding the preset parsing time), errors (e.g., parsing algorithm malfunction or parsing process interruption), or invalid parsing results (e.g., semantic contradictions, syntax errors, missing core elements) occur. When any of the above situations are detected, a degradation mechanism is triggered, and a preset template library is called. The preset template library stores predefined query templates corresponding to different query scenarios and intentions. The predefined query templates contain standardized semantic structures and fillable parameter positions. The predefined query template that best matches the current query scenario and intention is matched in the template library, and the extracted parameters are automatically filled into the corresponding parameter positions of the predefined query template to complete the generation of the semantic query statement and ensure the normal output of the semantic query statement.

[0057] In the above manner, when the large model re-ranking is abnormal or the confidence of the re-ranking result is insufficient, the candidate set ranked first in recall is used as the best candidate set, which avoids the process interruption caused by the re-ranking abnormality, ensures the normal acquisition of the best candidate set, and maintains the continuity of the query process. When the semantic parsing is abnormal, the semantic query statement is generated by pre-set template matching and parameter filling, which avoids the statement generation failure caused by the parsing abnormality and ensures the normal output of the semantic query statement.

[0058] Optionally, in some embodiments, please refer to Figure 3 The following is a block diagram of an intelligent dialogue analysis system based on multi-stage semantic matching, provided in an embodiment of this application. It includes a user interaction module, a large-scale model semantic processing module, a semantic model management module (pre-configured as a semantic model definition, knowledge base, and pattern mapper), a query construction and execution module, and a data visualization module, detailed below: This intelligent dialogue analysis system was deployed in a retail enterprise, whose data environment included more than 30 business datasets covering topics such as sales, inventory, and customers.

[0059] S1, through the user interaction module, enter the query in the Web (World Wide Web) interface: "Could you please show me the sales revenue rankings for each region last quarter? Thank you!" For example, it can receive a query statement and load the user's recent related historical queries for reference.

[0060] S2: Start the large model semantic processing module -S2.1 Problem Refinement and Optimization: The large model LLM identifies and removes noise such as "trouble" and "thank you", extracts the core intent "last quarter, major regions, sales revenue, ranking", and standardizes "major regions" into "sales areas".

[0061] -S2.2Embedding Fast Recall: The refined query "last quarter, sales regions, sales revenue, ranking" is vectorized and the cosine similarity is calculated with all pre-stored dataset vectors to quickly recall the Top-5 candidate set, including "regional sales summary table", "quarterly business report", "product sales details", etc.

[0062] -S2.3 Intelligent Reordering: LLM scores the 5 candidate sets based on dynamic weights (using "sales revenue" as an indicator, automatically increasing the field weight if detected). For example, the "Regional Sales Summary Table" was selected as the best dataset because it explicitly contains the fields "sales region", "quarter", and "sales revenue" and has the highest description matching degree.

[0063] S3: Execute the query construction execution module.

[0064] -S3.1 Semantic Parsing: The semantic parsing submodule combines the selected "Regional Sales Summary Table" and the mapping information provided by the schema mapper from the knowledge base. For example, "Sales Revenue" is mapped to the physical field sales_amount, and "Sales Region" is mapped to region, thereby generating S2SQL statements; [SELECT][region],[sales_amount][FROM] Regional Sales Summary [WHERE][quarter]='2024Q1' [ORDER BY][sales_amount][DESC] -S3.2 Semantic Correction: The S2SQL is checked through the correction submodule to confirm that the logic is reasonable (such as the existence of dimensions and metrics, and the correct aggregation relationship). If there are no errors, it is approved.

[0065] -S3.3 Physical SQL Construction and Optimization: Convert S2SQL into physical SQL for the target data warehouse, such as MySQL: SELECT region, SUM(sales_amount) AS total_sales FROM t_regional_sales WHERE quarter='2024Q1' GROUP BY region ORDER BY total_sales DESC The query optimizer recommends using indexes on the quarter and region fields.

[0066] S4: The data visualization module executes this SQL and retrieves the query results.

[0067] For example, based on the characteristics of the result data (the classification dimension "region" and the numerical indicator "sales revenue"), a bar chart is automatically selected for visualization rendering, clearly displaying the sales ranking of each region, and providing an interactive function to drill down to the province level.

[0068] The entire process, from query input to result display, maintains a response time within seconds, boasts high matching accuracy, and provides a smooth user experience. High availability is ensured through modular system design and multi-level degradation strategies, while the introduction of components such as a knowledge base, pattern mapper, and semantic corrector significantly improves the accuracy and robustness of SQL generation.

[0069] When a new large model is released, only the large model in the semantic processing module needs to be replaced, without changing the overall architecture; similarly, when an enterprise needs to access a new data source, it only needs to configure a new schema mapping in the semantic model management module.

[0070] Through the above methods, including refined two-stage dataset matching, context-enhanced semantic parsing, and multi-level verification and degradation mechanisms, a highly accurate, efficient, and available end-to-end processing model from natural language to data insights is achieved, providing strong technical support for enterprises to build intelligent data analysis platforms.

[0071] Please refer to Figure 2 This application also provides a structural block diagram of an intelligent dialogue analysis system based on multi-stage semantic matching, comprising: The acquisition module 210 is used to acquire user instructions, which are represented as text-based instructions. The semantic processing module 220 uses a large model to extract the intent of user commands, recalls multiple candidate sets based on the obtained target intent, and selects the candidate set with the highest score as the best candidate set. The semantic matching module 230 is used to call the semantic model to perform semantic parsing on natural language entities in the best candidate set and generate semantic query statements. The pre-built semantic model is used to form an index of the mapping relationship between natural language entities and physical data models in the knowledge base. The semantic correction module 240 is used to perform semantic correction on the semantic query statement, and to transform the corrected semantic query statement to determine the executable SQL statement; The query module 250 is used to query the data source based on executable SQL statements, determine the query results, and display them.

[0072] By employing the above methods, a large-scale model is used to extract user intent from commands. Based on the obtained target intent, multiple candidate sets are recalled, and the candidate set with the highest score is selected as the best candidate set. Through large-scale model intent extraction and precise candidate set selection, the accuracy of intent understanding and candidate matching precision are improved. A semantic model is invoked to perform semantic parsing on the natural language entities in the best candidate set, generating semantic query statements. The pre-built semantic model is used to index the mapping relationship between natural language entities and physical data models in the knowledge base. By establishing an entity-physical data model mapping index through the semantic model, semantic ambiguity is reduced and parsing reliability is improved. The semantic query statements are semantically corrected and transformed to determine executable SQL statements. Through semantic correction and SQL transformation, the statement error rate is reduced and the query execution stability is improved. The data source is queried based on the executable SQL statements, the query results are determined and displayed. Through standardized query and result display, end-to-end automation of natural language to database query is achieved, improving overall query efficiency and accuracy, and reducing the cost of manually writing SQL statements.

[0073] It should be noted that the intelligent dialogue analysis system and method based on multi-stage semantic matching provided in the above embodiments belong to the same concept. The specific methods of execution of each module have been described in detail in the method embodiments and will not be repeated here. In practical applications, the intelligent dialogue analysis method based on multi-stage semantic matching provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0074] In some embodiments, an electronic device is also provided, which may be a server, and its internal structure diagram is shown below. Figure 4As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. The computer program is executed by the processor to implement the functions or steps of the server-side method described above.

[0075] In some embodiments, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described above.

[0076] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0077] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or electronic device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0079] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. An intelligent dialogue analysis method based on multi-stage semantic matching, characterized in that, include: Obtain user instructions, which are represented as text instructions; The user instructions are extracted using a large model, multiple candidate sets are recalled based on the obtained target intent, and the candidate set with the highest score is selected as the best candidate set. The semantic model is invoked to perform semantic parsing on the natural language entities in the best candidate set, and a semantic query statement is generated. The pre-built semantic model is used to form an index of the mapping relationship between the natural language entities and the physical data model in the knowledge base. The semantic query statement is semantically corrected, and the corrected semantic query statement is transformed to determine the executable SQL statement; The executable SQL statement is used to query the data source, determine the query results, and display them.

2. The intelligent dialogue analysis method based on multi-stage semantic matching according to claim 1, characterized in that, The intent extraction of the user commands is performed using a large model, including: The user commands are cleaned using a large model. The data cleaning includes noise reduction and polite language filtering. The noise reduction includes data deduplication, missing value imputation, and outlier filtering. The intent of the user instructions after data cleaning is extracted using a large model. Based on the extracted intent, the corresponding semantic model is invoked to standardize and map the business terms in the intent to determine the target intent.

3. The intelligent dialogue analysis method based on multi-stage semantic matching according to claim 1, characterized in that, Based on the obtained target intent, multiple candidate sets are recalled, and the candidate set with the highest score is selected as the best candidate set, including: The target intent is converted into a query vector, the similarity between the query vector and the vector dataset represented by each dataset is calculated, and a preset number of datasets with the highest similarity are selected as multiple candidate sets. The candidate set is weighted using a large model to determine the candidate set with the highest score as the best candidate set. The weight of each candidate set is positively correlated with the fields, name and description in the candidate set.

4. The intelligent dialogue analysis method based on multi-stage semantic matching according to claim 1, characterized in that, The semantic model is constructed in the following ways: Configure a semantic model for the large model based on business parameters, including business themes, metrics, and dimensions. The semantic model is used to extract the mapping relationship between datasets, metrics, and dimensions, and an index is constructed and stored in a knowledge base. A pattern mapper is constructed to enable the natural language entities in the user instructions to be matched with terms and semantically associated in the knowledge base.

5. The intelligent dialogue analysis method based on multi-stage semantic matching according to claim 4, characterized in that, The semantic model is invoked to perform semantic parsing on the natural language entities in the best candidate set, generating a semantic query statement, including: The semantic model is invoked to find natural language entities in the best candidate set, and the physical data model in the knowledge base is matched with the corresponding context. Based on the optimal candidate set and the corresponding context, semantic parsing is performed to determine the semantic query statement.

6. The intelligent dialogue analysis method based on multi-stage semantic matching according to any one of claims 1 to 5, characterized in that, The semantic query statement is semantically corrected, and the corrected semantic query statement is transformed to determine an executable SQL statement, including: The semantic query statement is semantically corrected to determine the corrected query statement. The semantic correction includes at least one of the following: integrity check, logical consistency check, syntax legality check, and security check. The corrected query statement is converted into a physical query statement that conforms to the dialect of the target database and is determined to be an executable SQL statement.

7. The intelligent dialogue analysis method based on multi-stage semantic matching according to any one of claims 1 to 5, characterized in that, Before using a large model to extract intent from the user instructions, the process also includes: The user instructions, the semantic query statements, the executable SQL, and the query results are encapsulated in a structured manner to form a historical dialogue trajectory, which is then stored. In a new dialogue, when a natural language query with semantic similarity to the historical dialogue trajectory is detected, at least one historical interaction record that is identical to the current query domain, intent, metric, or dimension is recalled from the historical dialogue trajectory as a sample example. The sample example is concatenated with the current user's natural language query to form a prompt word.

8. The intelligent dialogue analysis method based on multi-stage semantic matching according to any one of claims 1 to 5, characterized in that, It also includes at least one of the following: If the large model intelligent rearrangement is abnormal or the confidence of the rearrangement result is still lower than the preset threshold, the candidate set ranked first in recall will be used as the best candidate set. If the semantic parsing of the large model times out, encounters an error, or the parsing result is invalid, a predefined query template is matched from the preset template library, and parameters are automatically filled in to generate a semantic query statement.

9. An intelligent dialogue analysis system based on multi-stage semantic matching, characterized in that, include: The acquisition module is used to acquire user instructions, which are represented as text-based instructions; The semantic processing module uses a large model to extract the intent of the user command, recalls multiple candidate sets based on the obtained target intent, and selects the candidate set with the highest score as the best candidate set. The semantic matching module is used to call the semantic model to perform semantic parsing on the natural language entities in the best candidate set and generate semantic query statements. The pre-built semantic model is used to form an index of the mapping relationship between the natural language entities and the physical data model in the knowledge base. The semantic correction module is used to perform semantic correction on the semantic query statement, and to transform the corrected semantic query statement to determine the executable SQL statement; The query module is used to query the data source based on the executable SQL statement, determine the query results, and display them.

10. An electronic device, characterized in that, It includes a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the intelligent dialogue analysis method based on multi-stage semantic matching as described in any one of claims 1 to 8.

Images