A knowledge query method and apparatus based on a large model

Abstract

This application provides a knowledge query method and apparatus based on a large model. The method includes: performing intent recognition on executable SQL sample code that has undergone syntax verification to obtain query intent-SQL code pairs, including the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code. This generates sample pairs that do not require manual verification using the verified SQL sample code, thereby increasing the number of sample pairs in the knowledge base. The query intent-SQL code pairs are then processed according to different dimensions to generate multi-dimensional sample pairs, further increasing the number of sample pairs. In this way, the SQL code corresponding to the user's needs can be determined based on the sample pairs in the knowledge base, and the retrieved SQL code can be used to guide the generation process of the large model, improving the accuracy and reliability of the generated results.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a knowledge query method and apparatus based on a large model. Background Technology

[0002] ChatBI utilizes Large Language Models (LLMs) to respond to user queries, such as generating SQL code or data analysis results. However, due to the inherent "illusion" problem of large models, the accuracy and reliability of ChatBI's generated results are relatively low. Currently, the generation process of large models is guided by knowledge relevant to the user's query information retrieved from the ChatBI knowledge base, aiming to improve the accuracy of ChatBI's generated results. The ChatBI knowledge base includes a large number of labeled sample pairs, each containing the user's query information and labeled content (i.e., SQL statements) related to that query.

[0003] However, when there are few or no labeled samples in the ChatBI knowledge base (for example, during a ChatBI cold start), it is impossible to retrieve knowledge related to the user's query from the ChatBI knowledge base, affecting the accuracy of ChatBI's generated results. Summary of the Invention

[0004] This application provides a knowledge query method, apparatus, device, storage medium, and program product based on a large model to solve the problem that relevant knowledge cannot be queried when there are few or no labeled samples in the ChatBI knowledge base in the prior art, thereby improving the accuracy of ChatBI generated results.

[0005] This application provides a knowledge query method based on a large model, including: Obtain the table creation statements corresponding to each table name in the SQL sample code in the target application database. Use the intent recognition model to perform intent recognition on each table creation statement to obtain the query intent-SQL code pair corresponding to the SQL sample code. The query intent-SQL code pair includes: the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code; the SQL sample code is executable code that has passed syntax verification. For each query intent-SQL code pair, corresponding processing is performed on the query intent-SQL code pair under different dimensions to obtain a multi-dimensional data set corresponding to the query intent-SQL code pair. Where the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the query intent-SQL code pair is processed according to the subquery extraction dimension. The processing corresponding to the subquery extraction dimension includes: generating at least two subquery sample pairs based on the query intent-SQL code pair; each subquery sample pair contains: a query indicator field contained in the query intent of the query intent-SQL code pair, and substandard SQL code generated based on the SQL code segment in the SQL code of the query intent-SQL code pair that corresponds to the query indicator field; the query indicator field in any subquery sample pair indicates a corresponding query intent. Among them, each query intent-SQL code pair corresponds to a multi-dimensional data set, which is used to query the corresponding SQL code based on user needs.

[0006] This application provides a knowledge query device based on a large model, including: The identification module is configured to obtain the table creation statements corresponding to each table name in the SQL sample code in the target application database, and use the intent recognition model to perform intent recognition on each table creation statement to obtain the query intent-SQL code pair corresponding to the SQL sample code. The query intent-SQL code pair includes: the query intent corresponding to the SQL sample code, and the standard SQL code corresponding to the SQL sample code; the SQL sample code is executable code that has passed syntax verification. The processing module is configured to perform corresponding processing on each query intent-SQL code pair under different dimensions to obtain a multi-dimensional data set corresponding to the query intent-SQL code pair. Specifically, when the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the query intent-SQL code pair is processed according to the subquery extraction dimension. The processing corresponding to the subquery extraction dimension includes: generating at least two subquery sample pairs based on the query intent-SQL code pair; each subquery sample pair contains: a query indicator field contained in the query intent of the query intent-SQL code pair, and substandard SQL code generated based on the SQL code segment in the SQL code of the query intent-SQL code pair that corresponds to the query indicator field; the query indicator field in any subquery sample pair indicates a corresponding query intent. Among them, each query intent-SQL code pair corresponds to a multi-dimensional data set, which is used to query the corresponding SQL code based on user needs.

[0007] This application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned knowledge query method based on a large model.

[0008] This application provides an electronic device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; wherein the processor is used to execute the machine-executable instructions to implement the above-described knowledge query method based on a large model.

[0009] This application provides a machine-readable storage medium storing machine-executable instructions that can be executed by a processor; wherein the processor executes the machine-executable instructions to implement the aforementioned knowledge query method based on a large model.

[0010] As can be seen from the above technical solutions, in this embodiment, intent recognition is performed on the executable SQL sample code that has undergone syntax verification to obtain query intent-SQL code pairs, including the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code. Thus, sample pairs that do not require manual verification are generated using the verified SQL sample code, increasing the number of sample pairs in the ChatBI knowledge base. Simultaneously, the query intent-SQL code pairs are processed according to different dimensions to generate multi-dimensional sample pairs, further increasing the number of usable sample pairs in the ChatBI knowledge base. In this way, the SQL code corresponding to the user's needs can be determined based on the sample pairs in the ChatBI knowledge base, and the retrieved SQL code can be used to guide the generation process of large models, improving the accuracy and reliability of the large model generation results. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a knowledge query method based on a large model provided in some embodiments of this application; Figure 2 This is a schematic diagram of the process for generating multi-dimensional data sets provided in some embodiments of this application; Figure 3 This is a flowchart illustrating the process of generating SQL sample code provided in some embodiments of this application; Figure 4 This is a schematic diagram of the workflow of the SQL conversion agent provided in some embodiments of this application; Figure 5 This is a flowchart illustrating the process of generating query intent-SQL code pairs provided by some embodiments of this application; Figure 6 These are schematic diagrams illustrating the process of processing query intent-SQL code pairs from different dimensions, provided by some embodiments of this application. Figure 7 This is a flowchart illustrating the process of generating reference intent-SQL code pairs provided by some embodiments of this application; Figure 8 This is a schematic diagram of the process for generating subquery sample pairs provided in some embodiments of this application; Figure 9 This is a schematic diagram of the process for generating standard query intents provided by some embodiments of this application; Figure 10 This is another schematic diagram illustrating the process of generating standard SQL query intents provided by some embodiments of this application; Figure 11 These are schematic diagrams of the structure of a knowledge query device based on a large model provided in some embodiments of this application; Figure 12 This is a hardware structure diagram of a knowledge query system based on a large model provided in some embodiments of this application. Detailed Implementation

[0012] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0013] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0014] In the description of embodiments of this application, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0015] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably. A model can also include different types of processing units or networks.

[0016] In one possible implementation, all data involved in this embodiment (such as the actual and predicted gaze points when the user device estimates the user's gaze, the device type of the user device, the user's user identifier, etc.) are only obtained and used with the knowledge and authorization of the relevant user.

[0017] As mentioned earlier, when ChatBI has few or no labeled samples in its knowledge base, it becomes impossible to retrieve relevant information from the ChatBI knowledge base, affecting the accuracy of ChatBI's generated results. Currently, most methods rely on manual annotation to obtain labeled samples, but this method is inefficient.

[0018] Therefore, this application provides a knowledge query method based on a large model. By performing intent recognition on executable SQL sample code that has undergone syntax validation, it obtains query intent-SQL code pairs, including the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code. This allows the generation of sample pairs without manual validation using the validated SQL sample code, increasing the number of sample pairs in the ChatBI knowledge base. Simultaneously, the query intent-SQL code pairs are processed according to different dimensions to generate multi-dimensional sample pairs, further increasing the number of usable sample pairs in the ChatBI knowledge base. In this way, the SQL code corresponding to the user's needs can be determined based on the sample pairs in the ChatBI knowledge base, and the retrieved SQL code can be used to guide the generation process of the large model, improving the accuracy and reliability of the generated results.

[0019] The following describes the knowledge query method based on a large model according to specific embodiments of this application. Figure 1 This is a flowchart illustrating a knowledge query method based on a large model provided in some embodiments of this application. This method can be implemented or included in devices or servers related to ChatBI.

[0020] like Figure 1As shown in box 101, the table creation statements corresponding to each table name in the SQL sample code are obtained in the target application database. The intent recognition model is used to perform intent recognition on each table creation statement to obtain the query intent-SQL code pair corresponding to the SQL sample code. The query intent-SQL code pair includes: the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code; the SQL sample code is executable code that has passed syntax verification.

[0021] In some embodiments, SQL code for different scenarios can be pre-collected as source SQL code. Source SQL code can include BI report SQL code, analyst SQL code, and big data platform SQL code. BI report SQL primarily generates the top-level SQL code for datasets and generally uses relational database dialects. Analyst SQL mainly handles daily data retrieval, analysis, and daily data report delivery, including both HIVE and relational database dialects. Big data platform SQL code primarily serves the daily output of data from data warehouse tables and mainly uses the HIVE dialect.

[0022] In some embodiments, the collected SQL source code can be cleaned to reduce introduced noise. For example, invalid comments, logout codes, and special placeholders in the collected SQL source code can be replaced with null values, and time variables in the collected SQL source code can be replaced with fixed values.

[0023] The target application database is the database corresponding to the SQL sample code. The SQL sample code can be executed in the target application database to meet the user's data query or data analysis needs. For example, if the SQL sample code is the collected raw SQL code, the target application database is the raw database corresponding to the raw SQL code. If the SQL sample code is SQL code obtained after dialect conversion of the raw SQL code based on ChatBI's downstream tasks, the target application database is the database corresponding to the downstream task.

[0024] The SQL sample code is executable code that has undergone syntax validation. For example, a large language model can be used to perform syntax validation on the collected SQL code to filter out SQL code with syntax errors. Simultaneously, the collected SQL code can be executed in the target database to filter out SQL code that cannot be executed. Furthermore, SQL code that fails validation or fails to execute can be corrected to obtain executable SQL sample code that has undergone syntax validation.

[0025] For a specific SQL sample code, determine the names of all the physical data tables included in that SQL sample code. Then, based on the target application knowledge base, determine the table creation statements used to create these data tables. SQL table creation statements are Data Definition Language (DDL) instructions used in relational databases to define and create a table structure. SQL table creation statements can declare the table name, column (field) names, data types, lengths, nullability, default values, primary keys, foreign keys, indexes, partitioning methods, and other table structure information all at once.

[0026] Based on the table creation statements corresponding to each table name in the SQL sample code and the generated prompt words, a large-scale intent recognition model is used to identify the intent of the SQL sample code based on the prompt words, thereby obtaining the query intent corresponding to the sample code. Based on the determined query intent, a query intent-SQL code pair is obtained. Standard SQL code is code that has undergone standardized processing of the SQL sample code; for example, syntax correction and dialect conversion can be performed on the original SQL code.

[0027] In box 102, for each query intent-SQL code pair, corresponding processing is performed on the query intent-SQL code pair under different dimensions to obtain a multi-dimensional data set corresponding to the query intent-SQL code pair. Specifically, when the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the query intent-SQL code pair is processed according to the subquery extraction dimension. The processing corresponding to the subquery extraction dimension includes: generating at least two subquery sample pairs based on the query intent-SQL code pair; each subquery sample pair contains: a query indicator field contained in the query intent of the query intent-SQL code pair, and sub-standard SQL code generated based on the SQL code segment in the SQL code of the query intent-SQL code pair that corresponds to the query indicator field. The multi-dimensional data set corresponding to each query intent-SQL code pair is used to query the corresponding SQL code based on user needs.

[0028] In some embodiments, for a given query intent-SQL code pair, the multi-dimensional dataset corresponding to that query intent-SQL code pair includes sample pairs across multiple dimensions, such as sample pairs under subquery extraction dimensions, sample pairs under similar derived dimensions, etc. The processing corresponding to the subquery extraction dimensions includes: determining multiple subquery intents based on the query intent in the query intent-SQL code pair, and determining code fragments corresponding to each subquery intent from the SQL code in the query intent-SQL code pair. Alternatively, determining multiple code fragments based on the SQL code in the query intent-SQL code pair, each code fragment being used to query a corresponding metric. Subsequently, determining the subquery intent corresponding to each code fragment from the query intent in the query intent-SQL code pair. Then, supplementing the code fragments based on the obtained subquery intents to generate SQL code that can be used to implement the subquery intent. Combining the supplemented SQL code and the subquery intent to form a subquery sample pair.

[0029] In some embodiments, the query intent-SQL code pair is processed from the dimension of query intent similarity to derive at least one reference intent-SQL code pair corresponding to the query intent-SQL code pair, wherein the query intent in any query intent-SQL code pair is different from the reference intent in each corresponding reference intent-SQL code pair.

[0030] Complex requirements are predefined requirements. For example, a complex requirement might include fields queried in the SQL sample code or query intent that do not directly appear in the data table, and whose values ​​need to be calculated using complex logic (e.g., conditional statements, aggregate functions, window functions, etc.). If the query intent or SQL code in the query intent-SQL code pair meets the complex requirement, then the query intent-SQL code pair is processed according to the subquery extraction dimension.

[0031] In some embodiments, after obtaining the multi-dimensional data set corresponding to the query intent-SQL code pair, the standard query intent for each sample pair in the knowledge base can also be determined. The original sample pair includes the collected original query intents and SQL codes when performing data queries against the target application database.

[0032] For each sample pair in the target dataset, the table creation statement corresponding to the table name in the SQL code of that sample pair is determined from the target application database. Prompt words are generated based on the SQL code and the table creation statement. Subsequently, a large language model is used to generate the SQL standard query intent corresponding to that sample pair based on the prompt words. Multiple SQL standard query intents corresponding to each sample pair are arranged in a specified order to generate an SQL standard query dictionary. The target dataset includes the multi-dimensional dataset obtained above (e.g., query intent-SQL code pairs, subquery sample pairs, and reference intent-SQL code pairs) and the original sample pairs. The original sample pairs include the collected original query intents and SQL codes used when performing data queries against the target application database.

[0033] In some embodiments, for a given query intent-SQL code pair or original sample pair, a candidate sample pair set corresponding to that query intent-SQL code pair or original sample pair can be determined first. The candidate sample pair set corresponding to a given sample pair includes sub-query sample pairs generated based on that sample pair and similar query samples generated based on that sample pair. If the semantic feature similarity between at least two sample pairs in the candidate sample pair set meets a set similarity requirement, then one of these at least two sample pairs is selected and the remaining sample pairs are deleted. Subsequently, based on the sample pairs in the candidate sample pair set whose semantic feature similarity does not meet the set similarity requirement, a multi-dimensional data set corresponding to the query intent-SQL code pair is obtained.

[0034] As can be seen, this application identifies the intent of executable SQL sample code that has undergone syntax verification, thereby obtaining query intent-SQL code pairs that include the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code. This allows for the generation of sample pairs without manual verification using the verified SQL sample code, increasing the number of sample pairs in the ChatBI knowledge base. Simultaneously, the query intent-SQL code pairs are processed according to different dimensions to generate multi-dimensional sample pairs, further increasing the number of usable sample pairs in the ChatBI knowledge base. In this way, the SQL code corresponding to user needs can be determined based on the sample pairs in the ChatBI knowledge base, allowing the retrieved SQL code to guide the generation process of large models, improving the accuracy and reliability of the generated results.

[0035] I. Generating Query Intent Based on SQL Sample Code Obtained from Dialect Conversion - SQL Code Pair In some embodiments, the type of database associated with ChatBI can be determined based on the application scenario of ChatBI or the tasks that ChatBI needs to perform, and that type of database can be used as the target application database. Subsequently, the collected raw SQL code is converted into sample SQL code that can be applied to the target application database to generate query intent-SQL code pairs corresponding to the target application database.

[0036] Figure 2 This is a schematic diagram of the process for generating multi-dimensional data sets provided in some embodiments of this application. Figure 3 This is a flowchart illustrating the process of generating sample SQL code provided in some embodiments of this application. For example... Figure 2 As shown in box 210, the collected SQL source code is first converted to SQL dialect.

[0037] like Figure 3 As shown in box 310, the raw SQL code is first obtained through data collection. This raw SQL code includes SQL code for BI reports, analyst SQL code, and big data platform SQL code. In box 320, the obtained raw SQL code undergoes SQL cleaning and database / table extraction. For example, regular expressions can be used to extract table names from the cleaned raw SQL code. For instance, regular expressions can be used to identify the names pointing to physical data tables from positions such as FROM / JOIN / INSERT INTO in the raw SQL code. Subsequently, based on the data lineage of the big data platform or by tracing the table names in the SQL code, the correspondence between the target table names in the target application database and the obtained table names is determined, thereby establishing the database-table mapping relationship.

[0038] In box 330, the SQL dialect conversion agent 330 is used to process the original SQL code to obtain the converted SQL reference code. The database table mapping relationship and the original SQL code are put into the system prompt of the SQL conversion agent, and the original SQL code is placed as a user message after the system prompt, forming a message list [{"role":"system", "content":system prompt of SQL conversion agent},{"role":"user", "content":cleaned original SQL code}]. Figure 4 This is a schematic diagram illustrating the workflow of the SQL conversion agent provided in some embodiments of this application. For example... Figure 4As shown, the system prompt word 410 of the SQL conversion agent and the cleaned original SQL code 420 are concatenated to form a message list 430. Then, the large model interface 440 is called, and the message list 430 is submitted to the large model, outputting the converted SQL reference code 450 in the target dialect format. Table 1 shows an example of the prompt words for the SQL dialect conversion agent 330.

[0039] Table 1 As shown in Table 1, ORI_DB represents the original database (e.g., HIVE database), and OBJ_DB represents the target application database (e.g., MySQL database, StarRocks database, etc.). TABLE_MAP represents the database-table mapping relationship, including five parts: original table name to target table name mapping, original field name to target field name mapping, renamed field mapping, deleted field list, and default filter conditions. When converting SQL code from the original database to the target application database, there may be changes in table names, field names, and the addition or deletion of fields. Therefore, providing the database-table mapping relationship to the large model allows the large model to directly replace and delete table names and field names during the SQL dialect conversion process, thus synchronizing table name changes to the executable production-level SQL code.

[0040] For example, the database-table mapping relationship may include: {"Original table name": "ads_bi_city_sale_order_fd"; "Target table name": "sr_bi_city_sale_order_fd"; "Rename field": [{"city_id": "city_code"}, {"order_id": "order_code"}, {"user_id": "user_code"}, {"dt": "create_time"}], "Delete field": ["oder_info", "city_level"], "Default filter condition": "user_code isnot null"}, etc.

[0041] Continue to refer to Figure 3In box 340, the SQL reference code 450 generated by the large model can be validated. For example, SQL reference code 450 can be executed in the target application database. In box 350, the SQL syntax can be determined based on the validation result. If the SQL syntax is correct, the SQL conversion is successful, and SQL reference code 450 is used as the SQL sample code. If a syntax error occurs during SQL validation, proceed to box 360. In box 360, it is determined whether the retry threshold has been reached. If the current iteration is less than the set number, the SQL validation error message is added to the SQL conversion agent prompt, and the SQL conversion is performed again, executing the steps shown in boxes 330, 340, and 350 until the SQL conversion is successful or the set maximum number of retries is reached.

[0042] Continue to refer to Figure 2 In box 220, SQL sample code obtained based on SQL dialect conversion is used to perform SQL2NL reverse annotation using a large model to obtain query intent-SQL code pairs. Figure 5 This is a flowchart illustrating the process of generating query intent-SQL code pairs provided in some embodiments of this application. For example... Figure 5 As shown, for each SQL sample code 510, table names can be extracted using regular expressions 520 to obtain a list of table names. The table creation statements corresponding to the table names in the SQL sample code are obtained from the target application database 530. Subsequently, system prompts 540 are generated based on the table creation statements and a pre-determined system prompt template. Based on the system prompts 540 and the SQL sample code 510, prompts (i.e., message list 550) used to determine the query intent are obtained. The message list can be in the following form: [{"role":"system":"content":system prompts for the SQL query target identification agent},{"role":"user":"content":dialect-converted SQL code}]. The prompts are then provided to the SQL reverse annotation agent 560 to obtain the output of the large model. The input to the large model can be the query intent, or a query intent-SQL code pair including the query intent. Table 2 shows an example of prompts used to determine the query intent.

[0043] Table 2 As shown in Table 2, the intent recognition model is used to summarize the query intent corresponding to the SQL sample code based on the query fields, special logic, filtering conditions, and table join logic in the SQL sample code, thereby generating query intent-SQL code pairs. The table creation statement information (TABLE_INFO) in Table 2 is used to provide contextual table structure information for reverse annotation of the large model.

[0044] In some embodiments, the SQL sample code can be SQL code that has not undergone dialect conversion. In this case, based on the original database corresponding to each SQL sample code, the table creation statement corresponding to the table name in each SQL sample code can be determined, thereby performing intent recognition on each SQL sample code based on the table creation statement to obtain query intent-SQL code pairs. Subsequently, the type of SQL code in the query intent-SQL code pairs can be converted based on the ChatBI task scenario. Further details are omitted here.

[0045] As can be seen, this embodiment utilizes a large-scale intent recognition model to process the SQL sample code to generate query intent-SQL code pairs. This eliminates the need for manual annotation of the query intent, improving the efficiency and reliability of sample pair generation. Furthermore, since the SQL sample code is syntactically validated and executable, manual verification is unnecessary, further enhancing the efficiency of sample pair generation.

[0046] II. Generating Reference Intent - SQL Code Pairs The original sample pairs collected manually mainly come from a small number of business personnel, operations personnel, and data analysts. Therefore, the expression style of the query intent in the original sample pairs and the query intent-SQL sample pairs generated based on the original SQL code is relatively professional and fixed. In downstream ChatBI applications, user input is characterized by personalization, colloquialism, and diverse expressions. In order to improve the recall rate of knowledge queries, sample pairs with different expression styles can be derived to increase the richness and diversity of the sample pairs.

[0047] like Figure 6 As shown in box 620, multiple reference intent-SQL code pairs corresponding to a given query intent-SQL code pair can be generated using a similar query-derived agent. The query intent of each reference intent-SQL code pair differs from the query intent tone or keywords in the original query intent-SQL code pair.

[0048] Figure 7 This is a flowchart illustrating the process of generating reference intent-SQL code pairs provided by some embodiments of this application. For example... Figure 7As shown, a message list 730 can be formed by combining pre-determined similar query-derived agent system prompts 710 and sample pairs 720. The message list 730 can be in the following form: [{"role":"system":"content":system prompts for the similar query-derived agent},{"role":"user":"content":sample pair}]. Subsequently, the message list 730 is provided to the large model through the large model interface 740 to determine the reference intent-SQL code pair 750 corresponding to the sample pair based on the large model output. Table 3 shows an example of prompts for the similar query-derived agent.

[0049] Table 3 As shown in Table 3, the similar query derivation agent is used to determine the keywords in the query intent of the query intent-SQL code pair. For example, if the query intent for a sample pair in data table A indicates a query for turnover (the product of selling price and sales volume), the keywords for this query intent can be determined to include "selling price," "sales volume," and "turnover." The similar query derivation agent can generate similar query intents with different tones or keywords based on the pre-determined query intent-SQL code pairs, and combine the query intents with the SQL codes in the query intent-SQL code pairs to form a reference intent-SQL code pair. For example, the keyword "turnover" determined above can be replaced with the similarly meaningful keyword "sales revenue."

[0050] As can be seen, this embodiment uses a similar query derivation method to generate samples with different expression styles, thereby improving the diversity and richness of the samples. Similarly, for the query intent-SQL code pairs generated by the reverse annotation of the large model, the large model is guided to perform text imitation through instructions to generate query intents with different language expression styles, thereby improving the diversity and richness of the samples. This, in turn, improves the recall rate and precision of the downstream ChatBI application's retrieval.

[0051] II. Generating Subsample Pairs In some application scenarios (e.g., multi-turn dialogue scenarios), knowledge base information is frequently used. As the number of dialogue rounds increases, user conversations tend to become fragmented, meaning that each user's dialogue content only contains partial information, and the focus of the user's questions changes with each round. The query intent-SQL code pair can be processed at the sub-extraction dimension to generate multiple subquery sample pairs. Figure 6 These are schematic diagrams illustrating the process of handling query intent-SQL code pairs from different dimensions, as provided in some embodiments of this application. For example... Figure 6As shown in box 610, a sample pair set 610 is defined, which includes all sample pairs in ChatBI's knowledge base. The sample pair set 610 includes all collected raw sample pairs and query intent-SQL code pairs obtained from reverse annotation. The raw sample pairs include the collected raw query intents and SQL codes used to execute data queries against the target application database.

[0052] In box 630, the Agent can use subqueries to process each sample pair in sample pair set 610 that meets the complexity requirement, in order to determine the subquery sample pair corresponding to each sample pair. The complexity requirement can be that the information queried by the sample pair does not appear in the data table and needs to be obtained through calculation. For example, if data table A includes order number, product type for each order, price of each product, and sales volume of each product. If the query intent or SQL code of a sample pair indicates a query for sales revenue (the product of price and sales volume), then that sample pair meets the complexity requirement. The complexity requirement can also be that the sample pair expects to query multiple pieces of information. For example, if the query intent or SQL code of a sample pair indicates a query for all order numbers and all product efficiencies, then that sample pair meets the complexity requirement.

[0053] Figure 8 This is a schematic diagram illustrating the process of generating subquery sample pairs provided in some embodiments of this application. For example... Figure 8 As shown, a message list 830 can be formed by combining the pre-determined subquery extraction Agent system prompt words 810 and each sample pair 820. The message list 830 can be in the following form: [{"role":"system":"content":system prompt words for the subquery extraction Agent},{"role":"user":"content":sample pair}]. Subsequently, the message list 830 is provided to the large model through the large model interface 840 to determine the subquery sample pair 850 corresponding to the sample pair based on the output of the large model. Table 4 shows an example of the prompt words for the subquery extraction Agent.

[0054] Table 4 As shown in Table 4, the Agent system prompt word 810 indicates that multiple subquery intentions (or calculated metrics) are determined from the query intention, and SQL fragments corresponding to each subquery intention are extracted from the SQL code, or multiple SQL fragments are extracted from the SQL code and the subquery intention corresponding to each SQL fragment is determined from the query intention. For example, if the query intention and SQL code for a sample pair in data table A indicate querying turnover (the product of selling price and sales volume), three subquery intentions can be determined based on the query intention of that sample pair: "query the selling price of each product," "query the efficiency of each product," and "calculate the product of selling price and sales volume of each product to determine turnover," and the corresponding SQL code fragment for each subquery intention can be determined. Furthermore, three code fragments can be determined based on the SQL sample code of that sample pair: "code fragment for querying product selling price," "code fragment for querying product sales volume," and "code fragment for calculating turnover," and the corresponding subquery intention for each code fragment can be determined. Subsequently, the determined code fragments are supplemented to make them complete SQL code.

[0055] Continue to refer to Figure 6 In box 640, the SQL code in the generated subquery sample pairs 850 can be validated to filter out subquery sample pairs that do not conform to syntax rules or cannot be executed. In box 650, sample deduplication can be performed on the determined subquery sample pairs and reference intent-SQL code pairs. For example, general text similarity can be used to determine the query intent similarity (e.g., vector cosine similarity) between a given query intent-SQL code pair and multiple subquery sample pairs and multiple reference intent-SQL code pairs obtained based on that query intent-SQL code pair. Subsequently, sample pairs with similarity exceeding a similarity threshold are merged into a single sample pair. This filters out highly similar samples, avoids too many duplicate samples in the knowledge base, and reduces redundant storage.

[0056] As can be seen, in this embodiment, multiple subquery sample pairs are constructed based on query intent-SQL code pairs to ensure that each retrieval process in a multi-round session scenario can find samples with higher matching degree, improve the recall rate, and thus better assist downstream tasks such as ChatBI intent recognition or SQL generation.

[0057] IV. Generating Standard Query Intent Figure 9 This is a schematic diagram of the process for generating standard query intents provided by some embodiments of this application. For example... Figure 9As shown, in box 910, the sample pair set is determined (including raw sample pairs collected manually, query intent-SQL code pairs, reference intent-SQL code pairs, and subquery sample pairs, etc.). In box 920, for each sample pair in the sample pair set, regular expressions are used to extract the table names from the SQL code of each sample pair. In box 930, the target application database is queried to obtain the table creation statements corresponding to each table name. In box 940, a prompt word is generated based on the obtained table creation statement, the system prompt word of the SQL standard query extraction agent, and the SQL code in the sample pair. In box 950, the prompt word is provided to the SQL standard query extraction agent to obtain multiple query intent fields, such as query fields, special calculation logic, filtering conditions, and association conditions. In box 960, the obtained multiple fields are concatenated to obtain the standard query intent corresponding to the sample pair.

[0058] Figure 10 This is another schematic diagram illustrating the process of generating standard SQL query intents provided by some embodiments of this application. For example... Figure 10 As shown, a message list 830 can be formed by combining a pre-determined SQL standard query intent Agent system prompt word 1010 and a sample pair 1020. The message list 1030 can be in the form of [{"role":"system":"content":system prompt word for the subquery extraction Agent}, {"role":"user":"content":SQL code of the sample pair}]. Subsequently, the message list 1030 is provided to the large model through the large model interface 1040 to determine the SQL standard query intent 1050 corresponding to the sample pair based on the large model output. Table 5 shows an example of a prompt word for the subquery extraction Agent.

[0059] Table 5 As shown in Table 5, prompts are provided to the large model by calling the large model interface, which then generates intent fields such as query table name, user question, query fields, special logic, and related conditions. Except for the query table name, all other parts are concatenated sequentially to form the standard SQL query intent. Subsequently, the SQL-annotated query dictionary is concatenated into a string in sequence.

[0060] Furthermore, a FAQ question-and-answer data structure serving downstream ChatBI retrieval tasks can be designed with sample pairs as the smallest data storage granularity. This data granularity ensures that the sample pairs obtained by each downstream ChatBI retrieval task are independent and complete, and can be directly embedded into the context of the large model as "expert experience." In the FAQ question-and-answer data structure, the question (similar_query) refers to the user's query intent (or standard SQL query intent), and the answer refers to the SQL code in the corresponding sample pair. The original sample pairs, query intent-SQL code pairs, subquery sample pairs, and reference intent-SQL code pairs are all stored in a general vector database (e.g., ElasticSearch, Milvus) in the form of an FAQ question-and-answer data structure. When performing intent recognition tasks or SQL generation tasks, the similarity between the user query information and the question field is calculated to obtain the corresponding sample pairs.

[0061] As can be seen, in this embodiment, the query intent in each sample pair is converted into a semi-structured SQL standard query intent to clarify the query fields, special calculation logic, filtering conditions, and association conditions in the SQL generation process. This SQL standard query intent is similar to the thought process of a human data analyst writing SQL, thereby providing a reference for downstream ChatBI tasks to generate SQL code based on user intent.

[0062] Based on the same concept as the above method, this application proposes a knowledge query device based on a large model, see [link to relevant documentation]. Figure 11The diagram shows a schematic of a knowledge query device based on a large model. The device may include: an identification module 111, configured to obtain the table creation statements corresponding to each table name in the SQL sample code in the target application database, and to use the intent recognition large model to perform intent recognition on each table creation statement to obtain the query intent-SQL code pair corresponding to the SQL sample code. The query intent-SQL code pair includes: the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code; the SQL sample code is executable code that has passed syntax verification. Processing module 112 is configured to perform corresponding processing on each query intent-SQL code pair under different dimensions to obtain a multi-dimensional data set corresponding to the query intent-SQL code pair. Specifically, when the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the query intent-SQL code pair is processed according to the subquery extraction dimension. The processing corresponding to the subquery extraction dimension includes: generating at least two subquery sample pairs based on the query intent-SQL code pair; each subquery sample pair includes: a query indicator field contained in the query intent of the query intent-SQL code pair, and sub-standard SQL code generated based on the SQL code segment in the SQL code of the query intent-SQL code pair that corresponds to the query indicator field; wherein, the multi-dimensional data set corresponding to each query intent-SQL code pair is used to query the corresponding SQL code based on user requirements.

[0063] Optionally, the device also includes a sample code acquisition module, which is configured to obtain target table names that have a mapping relationship with each original table name in the SQL source code in the target application database; generate SQL reference code corresponding to the SQL source code based on each original table name in the SQL source code and the target table names that have a mapping relationship with each original table name; execute the SQL reference code; and if the SQL reference code is successfully executed, the SQL reference code is the SQL sample code.

[0064] Optionally, the SQL sample code acquisition module is also configured to use the original table names in the SQL source code and the target table names that have a mapping relationship with each original table name as system prompt words, and combine them with the SQL source code to form a message; input each message into the large language model to obtain the SQL reference code corresponding to the SQL source code in each message.

[0065] Optionally, the apparatus further includes a derivation module, which is configured to, when the query intent or SQL code in the query intent-SQL code pair meets the complex requirements, perform query intent similarity processing on the query intent-SQL code pair from the dimension of query intent similarity to derive at least one reference intent-SQL code pair corresponding to the query intent-SQL code pair; the query intent in any query intent-SQL code pair is different from the reference intent in each corresponding reference intent-SQL code pair.

[0066] Optionally, the device further includes an intent extraction module, which is configured to, for each sample pair in the target dataset, obtain the table creation statements corresponding to each table name in the SQL code of the sample pair from the target application database, input the table creation statements and the SQL code in the sample pair into the SQL annotation query extraction Agent to obtain the SQL standard query intent corresponding to the sample pair; wherein, the target dataset includes the following sample pairs: original sample pairs, and sample pairs in the multi-dimensional dataset corresponding to each query intent-SQL code pair; the original sample pairs include the collected original query intents and SQL codes when performing data queries on the target application database; wherein, the query intent in each sample pair, and the SQL standard query intent corresponding to each sample pair, are used to query the corresponding SQL code based on user needs.

[0067] Optionally, the standard SQL query intent corresponding to any sample pair can be obtained by concatenating the user question, query fields, specified special logic, and relationships contained in the SQL code of the sample pair.

[0068] Optionally, the device also includes a reference code generation module, which is configured to update the iteration round count if the SQL reference code fails to execute; if the current iteration round count is less than a set number, generate SQL reference code corresponding to the SQL original code based on the reason for the SQL reference code's execution failure, the original table names in the SQL original code, and the target table names that have a mapping relationship with each original table name, and return to the step of executing the SQL reference code.

[0069] Optionally, the processing module is further configured to obtain a set of candidate sample pairs corresponding to the query intent-SQL code pair; the set of candidate sample pairs includes sample pairs obtained by processing the query intent-SQL code pair in different dimensions; if the semantic feature similarity between at least two sample pairs in the set of candidate sample pairs meets the set similarity requirement, then one sample pair is selected from the at least two sample pairs and the remaining sample pairs are deleted; based on each sample pair in the set of candidate sample pairs whose semantic feature similarity does not meet the set similarity requirement, a multi-dimensional data set corresponding to the query intent-SQL code pair is obtained.

[0070] Based on the same concept as the methods described above, this application proposes a knowledge query system based on a large model. (See [link to relevant documentation]). Figure 12 As shown, the knowledge query system based on the large model includes: a processor 121 and a machine-readable storage medium 122. The machine-readable storage medium 122 stores machine-executable instructions that can be executed by the processor 121. The processor 121 is used to execute the machine-executable instructions to implement the above-mentioned knowledge query method based on the large model.

[0071] Based on the same concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of computer instructions. When the computer instructions are executed by a processor, they can realize the knowledge query method based on a large model disclosed in the above examples of this application.

[0072] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0073] Based on the same application concept as the above method, this application embodiment also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-described knowledge query method based on a large model.

[0074] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A knowledge retrieval method based on a large model, characterized in that, The method includes: The table creation statements corresponding to each table name in the SQL sample code are obtained from the target application database. The intent recognition model is used to perform intent recognition on the SQL sample code based on each table creation statement to obtain the query intent-SQL code pair corresponding to the SQL sample code. The query intent-SQL code pair includes: the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code; the SQL sample code is executable code that has passed syntax verification. For each query intent-SQL code pair, corresponding processing is performed on the query intent-SQL code pair under different dimensions to obtain a multi-dimensional data set corresponding to the query intent-SQL code pair. Where the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the query intent-SQL code pair is processed according to the subquery extraction dimension. The processing corresponding to the subquery extraction dimension includes: generating at least two subquery sample pairs based on the query intent-SQL code pair; each subquery sample pair includes: a query indicator field contained in the query intent of the query intent-SQL code pair, and substandard SQL code generated based on the SQL code segment in the SQL code of the query intent-SQL code pair that corresponds to the query indicator field; the query indicator field in any subquery sample pair indicates a corresponding query intent. Among them, each query intent-SQL code pair corresponds to a multi-dimensional data set, which is used to query the corresponding SQL code based on user needs.

2. The method according to claim 1, characterized in that, The SQL sample code was obtained through the following steps: Obtain target table names that have a mapping relationship with each original table name in the SQL source code from the target application database. Generate SQL reference code corresponding to the SQL source code based on each original table name in the SQL source code and the target table names that have a mapping relationship with each original table name. Execute the SQL reference code. If the SQL reference code is executed successfully, then the SQL reference code is the SQL sample code.

3. The method according to claim 2, characterized in that, The step of generating SQL reference code corresponding to the original SQL code using a large language model based on the original table names in the original SQL code and the target table names that have a mapping relationship with each original table name includes: The original table names in the SQL source code, and the target table names that have a mapping relationship with each original table name, are used as system prompt words, which are combined with the SQL source code to form a message. Each message is input into the large language model to obtain the SQL reference code corresponding to the SQL source code in each message.

4. The method according to claim 1, characterized in that, When the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the method further includes: From the perspective of query intent similarity, the query intent-SQL code pair is processed according to the query intent similarity to derive at least one reference intent-SQL code pair corresponding to the query intent-SQL code pair; the query intent in any query intent-SQL code pair is different from the reference intent in each corresponding reference intent-SQL code pair.

5. The method according to claim 4, characterized in that, The method further includes: For each sample pair in the target dataset, the table creation statements corresponding to each table name in the SQL code of the sample pair are obtained from the target application database. The table creation statements and the SQL code of the sample pair are then input into the SQL annotation query extraction Agent to obtain the SQL standard query intent corresponding to the sample pair. The target dataset includes the following sample pairs: original sample pairs and sample pairs in the multi-dimensional dataset corresponding to each query intent-SQL code pair. The original sample pairs include the collected original query intents and SQL codes when performing data queries on the target application database. The query intent in each sample pair, and the corresponding standard SQL query intent for each sample pair, are used to retrieve the corresponding SQL code based on user needs.

6. The method according to claim 5, characterized in that, The standard SQL query intent corresponding to any sample pair is obtained by concatenating the user question, query fields, specified special logic, and relationships contained in the SQL code of that sample pair.

7. The method according to claim 2, characterized in that, If the SQL reference code fails to execute, the method further includes: Update iteration round count; If the current iteration round count is less than the set count, then based on the reason for the SQL reference code execution failure, the original table names in the original SQL code, and the target table names that have a mapping relationship with each original table name, generate SQL reference code corresponding to the original SQL code, and return to the step of executing the SQL reference code.

8. The method according to claim 1, characterized in that, The process of processing the query intent-SQL code pair under different dimensions to obtain the multi-dimensional data set corresponding to the query intent-SQL code pair includes: For the query intent-SQL code pair, obtain a set of candidate sample pairs corresponding to the query intent-SQL code pair; the set of candidate sample pairs includes sample pairs obtained by processing the query intent-SQL code pair in different dimensions. If the semantic feature similarity between at least two sample pairs in the candidate sample pair set meets the set similarity requirements, then select one from the at least two sample pairs and delete the remaining sample pairs; Based on the sample pairs in the candidate sample pair set whose semantic feature similarity does not meet the set similarity requirements, a multi-dimensional data set corresponding to the query intent-SQL code pair is obtained.

9. A knowledge query device based on a large model, characterized in that, The device includes: The identification module is configured to obtain the table creation statements corresponding to each table name in the SQL sample code in the target application database, and use the intent recognition model to perform intent recognition on each table creation statement to obtain the query intent-SQL code pair corresponding to the SQL sample code. The query intent-SQL code pair includes: the query intent corresponding to the SQL sample code and the standard SQL code corresponding to the SQL sample code; the SQL sample code is executable code that has passed syntax verification. The processing module is configured to perform corresponding processing on each query intent-SQL code pair under different dimensions to obtain a multi-dimensional data set corresponding to the query intent-SQL code pair. Specifically, when the query intent or SQL code in the query intent-SQL code pair meets complex requirements, the query intent-SQL code pair is processed according to the subquery extraction dimension. The processing corresponding to the subquery extraction dimension includes: generating at least two subquery sample pairs based on the query intent-SQL code pair; each subquery sample pair includes: a query indicator field contained in the query intent of the query intent-SQL code pair, and substandard SQL code generated based on the SQL code segment in the SQL code of the query intent-SQL code pair that corresponds to the query indicator field; the query indicator field in any subquery sample pair indicates a corresponding query intent. Among them, each query intent-SQL code pair corresponds to a multi-dimensional data set, which is used to query the corresponding SQL code based on user needs.

10. The apparatus according to claim 9, characterized in that, The device further includes a sample code acquisition module, which is configured to obtain target table names that have a mapping relationship with each original table name in the SQL original code in the target application database, and generate SQL reference code corresponding to the SQL original code based on each original table name in the SQL original code and the target table names that have a mapping relationship with each original table name. Execute the SQL reference code. If the SQL reference code is executed successfully, then the SQL reference code is the SQL sample code. And / or, The SQL sample code acquisition module is further configured to use the original table names in the SQL original code and the target table names that have a mapping relationship with each original table name as system prompt words, and combine them with the SQL original code to form a message; input each message into the large language model to obtain the SQL reference code corresponding to the SQL original code in each message; And / or, The device further includes a derivation module, which is configured to, when the query intent or SQL code in the query intent-SQL code pair meets the complex requirements, perform the query intent similarity processing on the query intent-SQL code pair from the dimension of query intent similarity, so as to derive at least one reference intent-SQL code pair corresponding to the query intent-SQL code pair. The query intent in any query intent-SQL code pair is different from the reference intent in each corresponding reference intent-SQL code pair; And / or, The device further includes an intent extraction module, which is configured to, for each sample pair in the target dataset, obtain the table creation statements corresponding to each table name in the SQL code of the sample pair from the target application database, input the table creation statements and the SQL code of the sample pair into the SQL annotation query extraction Agent, and obtain the SQL standard query intent corresponding to the sample pair; wherein, the target dataset includes the following sample pairs: original sample pairs, and sample pairs in the multi-dimensional dataset corresponding to each query intent-SQL code pair; the original sample pairs include the collected original query intents and SQL codes when performing data queries against the target application database; wherein, the query intents in each sample pair, and the corresponding SQL standard query intents, are used to query the corresponding SQL code based on user needs; and / or, The standard SQL query intent corresponding to any sample pair is obtained by concatenating the user question, query fields, specified special logic, and relationships contained in the SQL code of that sample pair; and / or, The apparatus further includes a reference code generation module, configured to update the iteration round count if the SQL reference code fails to execute; if the current iteration round count is less than a set number, generate SQL reference code corresponding to the SQL original code based on the reason for the SQL reference code's execution failure, the original table names in the original SQL code, and the target table names that have a mapping relationship with each original table name, and return to the step of executing the SQL reference code; and / or, The processing module is further configured to obtain a set of candidate sample pairs corresponding to the query intent-SQL code pair; the set of candidate sample pairs includes sample pairs obtained by processing the query intent-SQL code pair in different dimensions; if the semantic feature similarity between at least two sample pairs in the set of candidate sample pairs meets the set similarity requirement, then one sample pair is selected from the at least two sample pairs and the remaining sample pairs are deleted; based on each sample pair in the set of candidate sample pairs whose semantic feature similarity does not meet the set similarity requirement, a multi-dimensional data set corresponding to the query intent-SQL code pair is obtained.

Images