A DSL sentence-based reference example determination method and electronic device

Abstract

The application discloses a reference example determination method based on DSL statements and electronic equipment, and relates to the technical field of artificial intelligence. The application can acquire a plurality of historical data containing historical query statements, historical DSL statements, historical SQL statements and historical execution results; analyze semantics and operation types of a target query to obtain an intermediate query corresponding to the target query; determine a first candidate example by using the intermediate query and the historical DSL statements; perform quality checking processing by using the historical DSL statements in the first candidate example and the historical SQL statements to determine a second candidate example; and sort and expand the second candidate example based on example weights of the second candidate example and the historical execution results to obtain a reference example. By the method of the application, a high-quality reference example matched with a target query can be determined from historical data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a reference example determination method based on DSL statements and an electronic device. BACKGROUND

[0002] NL2SQL is a technology for converting natural language into structured query language (SQL), and DSL statements are a special language for a specific field, which are usually used as an intermediate layer in NL2SQL. That is, natural language is first converted into DSL statements, and then the DSL statements are converted into SQL statements. This way can handle more complex queries and has strong field adaptability, and has been widely used in report generation, data analysis, data query and other scenarios.

[0003] Currently, a large language model can be guided by a small number of examples to convert natural language into DSL statements, and then convert the DSL statements into SQL statements. When selecting examples, the mainstream scheme selects similar examples by comparing the coincidence degree of keywords and the structural similarity of syntax trees. However, these examples usually have not been quality checked, and the scene coverage is insufficient. In addition, the randomness of sorting will lead to low quality of these examples, making it difficult to select high-quality examples, so it is difficult to guide the large language model to generate accurate SQL statements based on examples.

[0004] Therefore, how to accurately determine high-quality reference examples is a technical problem to be solved. SUMMARY

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In one aspect, the present application provides a reference example determination method based on DSL statements, which comprises:

[0008] Obtaining a target query and a plurality of historical data, the historical data comprising historical query statements, historical DSL statements, historical SQL statements and historical execution results having an association relationship;

[0009] Analyzing the semantics and operation type of the target query to obtain an intermediate query corresponding to the target query;

[0010] Determining a first candidate example from the plurality of historical data using the intermediate query and the historical DSL statements;

[0011] The first candidate example is subjected to quality checking processing by using the historical DSL statement in the first candidate example and the historical SQL statement, so as to determine a second candidate example.

[0012] The second candidate example is subjected to sorting and expansion processing based on the example weight and the historical execution result of the second candidate example, so as to obtain the reference example corresponding to the target query.

[0013] In another aspect, the present application also provides a reference example determination device based on a DSL statement, which comprises:

[0014] An acquisition module is configured to acquire a target query and a plurality of historical data, wherein the historical data comprises historical query statements, historical DSL statements, historical SQL statements and historical execution results having an association relationship;

[0015] An analysis module is configured to analyze the semantics and the operation type of the target query, so as to obtain an intermediate query corresponding to the target query;

[0016] A candidate module is configured to determine a first candidate example from the plurality of historical data by using the intermediate query and the historical DSL statement;

[0017] A checking module is configured to subject the first candidate example to quality checking processing by using the historical DSL statement in the first candidate example and the historical SQL statement, so as to determine a second candidate example;

[0018] A reference module is configured to subject the first candidate example to sorting and expansion processing based on the historical DSL statement and the historical SQL statement in the second candidate example, so as to obtain a reference example corresponding to the target query.

[0019] In another aspect, the present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads the instructions from the memory to execute the steps in any of the reference example determination methods based on a DSL statement provided by the present application.

[0020] In another aspect, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in any of the reference example determination methods based on a DSL statement provided by the present application.

[0021] In another aspect, the present application also provides a computer program product comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps in any of the reference example determination methods based on a DSL statement provided by the present application.

[0022] The technical scheme provided by the present application has at least the following beneficial effects:

[0023] In the embodiment of the present application, the semantics and operation type of the target query are analyzed to generate an intermediate query with clearer semantics; the first candidate example is preliminarily determined from historical data by using the intermediate query and the historical DSL statement; the first candidate example is subjected to quality checking processing by using the historical DSL statement of the first candidate example and the historical SQL statement to determine a second candidate example with high quality; and the second candidate example is subjected to sorting and expansion processing based on the example weight of the second candidate example and the historical execution result, so as to ensure the arrangement order and scene coverage of the reference example and determine the high-quality reference example. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is an application scenario diagram of the reference example determination method based on the DSL statement provided by the embodiment of the present application;

[0026] Figure 2 is a flow diagram of the reference example determination method based on the DSL statement provided by the embodiment of the present application;

[0027] Figure 3 is a flow diagram of the quality checking processing provided by the embodiment of the present application;

[0028] Figure 4 is a structural diagram of the reference example determination device based on the DSL statement provided by the embodiment of the present application;

[0029] Figure 5 is a structural diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0031] It can be understood that in the specific embodiments of the present application, the data related to user information and the like need to obtain the permission or consent of the user, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the country and region.

[0032] Reference can be made to Figure 1 , which shows an application scenario diagram of the reference example determination method based on DSL statements. The application scenario can include a terminal 101 and a server 102, and the terminal 101 and the server 102 can exchange data through a network. The terminal 101 can be installed with an application program related to question and answer. The terminal 101 can be a mobile phone, a tablet computer, a smart Bluetooth device, a computer, a large screen device, a robot, etc. The server 102 can be a single server or a server cluster composed of multiple servers.

[0033] The user can send a target query to the server 102 through the terminal 101, so that the server 102 determines the reference example by executing the method of the present application.

[0034] The server 102 can obtain the target query and a plurality of historical data. The historical data can include historical query statements, historical DSL statements, historical SQL statements and historical execution results having an association relationship. The server 102 can analyze the semantics and operation type of the target query to obtain an intermediate query corresponding to the target query. The intermediate query and the historical DSL statements are used to determine a first candidate example from the plurality of historical data. The first candidate example is subjected to quality checking processing based on the historical DSL statements and the historical SQL statements in the first candidate example to determine a second candidate example. The second candidate example is sorted and expanded by using the example weight of the second candidate example and the historical execution result to obtain the final reference example.

[0035] Optionally, the reference example can be sent by the server 102 to the terminal 101 for display to the user for confirmation.

[0036] In the present application, a reference example determination method based on DSL statements is provided, as shown in Figure 2 The specific process of the reference example determination method based on DSL statements can be as follows:

[0037] S110, obtaining a target query and a plurality of historical data.

[0038] The target query is a natural language query input by a current user. The historical data refers to the queries and data related to the queries obtained in a historical time period. The historical time period refers to a time period before the current time, for example, one month before the current time, one week before the current time, etc.

[0039] It should be noted that in the NL2SQL technology, a natural language sentence can be converted into a structured query language (SQL) sentence to realize adding, deleting, modifying and inquiring of a database, and is a bridge connecting an application and a database, and is widely applied to intelligent report generation, enterprise data analysis, data query and the like. A domain-specific language (DSL) sentence is an intermediate layer of a natural language and a SQL sentence, has simple syntax and clear semantics, and can be used for accurately mapping a natural language intention and a SQL logic.

[0040] Each time a natural language query is received, the natural language query can be converted into a DSL sentence, the DSL sentence is converted into a SQL sentence, finally the SQL sentence is executed to obtain an execution result, and response data corresponding to the natural language query is generated based on the execution result.

[0041] The historical data in the embodiment of the application can include historical queries, historical DSL sentences, historical SQL sentences and historical execution results having an association relationship. The historical queries are natural language queries received in a historical time period, the historical DSL sentences are obtained by conversion of the historical queries, the historical SQL sentences are generated based on the historical DSL sentences, and the historical execution results are results obtained after the historical SQL sentences are executed.

[0042] The plurality of historical data can be pre-collected and stored in a specified database, and when subsequent use is required, the specified database can be directly accessed.

[0043] S120, analyze the semantics and operation type of the target query to obtain an intermediate query corresponding to the target query.

[0044] For the target query currently obtained, in order to clarify the semantic requirement of the target query, the target query can be analyzed and processed, the core semantics contained in the target query can be extracted, and the operation type corresponding to the operation that needs to be performed to realize the target query can be analyzed.

[0045] The intermediate query is the content obtained after the target query is analyzed, and the intermediate query can contain clearer semantics and operation types that can be required to complete the query.

[0046] As an implementation manner, when the semantics and operation type of the target query are analyzed to obtain the intermediate query, a prompt word template and a large language model can be used. The prompt word template can be set in advance, and the prompt word template can include the content that the large language model needs to analyze from the target query. For example, the prompt word template can be "core semantics that the target query needs to include: [], and operations that need to be covered: []", which successively guides the large language model to analyze the core semantics and the operation type, and then obtains the intermediate query.

[0047] The prompt word template is input into the large language model, so as to guide the large language model to analyze and infer the target query and extract the core semantics of the target query. In order to complete the task in the target query, the operation that needs to be performed is analyzed, for example, the target query is "count the number of employees in A department in 2024", the large language model analyzes that in order to complete the task, the year and the department need to be filtered through Filter, and the number of employees needs to be aggregated through Agg. The core semantics and the operation type after analysis are filled into the prompt word template, and the final intermediate query is obtained.

[0048] As shown in the foregoing example, the corresponding intermediate query is "core semantics that the target query needs to include: count the number of employees in A department in 2024, and operations that need to be covered: Filter (filter hire_year=2024, department=A department), Agg (COUNT (employee_id))".

[0049] S130, determining a first candidate example from the plurality of historical data by using the intermediate query and the historical DSL statement.

[0050] Compared with the target query, the intermediate query has a clearer and unambiguous semantics, and includes possible operation types. Using the intermediate query to determine the reference example can improve the accuracy of the reference example. The historical DSL statement in the historical data is also concise in syntax, and includes operation types, operation parameters, and associated logic. In order to fully utilize the historical data, the first candidate example can be selected based on the intermediate query and the historical DSL statement for subsequent use.

[0051] As an implementation manner, when the intermediate query and the historical DSL statement are used to determine the first candidate example from the plurality of historical data, the cosine similarity between the historical DSL statement and the intermediate query can be calculated for each historical DSL statement. Based on the cosine similarity, a preset number of historical DSL statements are selected as first DSL statements. The second DSL statement is determined in combination with the cosine similarity and the operation type in the first DSL statement. The historical data to which the second DSL statement belongs is determined as the first candidate example.

[0052] In the plurality of historical data, a plurality of corresponding historical DSL statements can be obtained, and each historical DSL statement is encoded by using a pre-trained DSL encoder to obtain a historical DSL vector corresponding to the historical DSL statement. Similarly, the intermediate query is encoded by using the DSL encoder to obtain a query vector. The DSL encoder can be obtained based on a BERT model fine-tuning, and the input data of the DSL encoder is a DSL statement, and the output of the DSL encoder is a feature vector corresponding to the DSL statement.

[0053] The cosine similarity between the query vector and each historical DSL vector is calculated, and all historical DSL statements are arranged in descending order of cosine similarity to obtain a historical DSL sequence. Then, the top k historical DSL statements in the historical DSL sequence are taken as the first DSL statement. Wherein, k is a positive integer, which can be set according to actual needs. In the few-shot learning paradigm, k is usually less than 5.

[0054] After screening the first DSL statement, in order to ensure the structural diversity of DSL in the first candidate example, it can be detected whether the operation type in the first DSL statement is repeated; if there is repetition, the repeated item with the highest cosine similarity is retained to replace the repeated item with lower cosine similarity. Wherein, the operation type in the specified DSL statement is different from the operation type in the first DSL statement, and the cosine similarity of the specified DSL statement is the highest among the corresponding operation types, so as to obtain k historical DSL statements with different operation types, as the second DSL statement.

[0055] Optionally, when determining the specified DSL statement, it can be detected from the historical DSL sequence, from the k+1th historical DSL statement, whether the operation type of the historical DSL statement is the same as that of the first DSL statement; if the same, the next one is detected; if different, it is taken as the specified DSL statement. In this way, a plurality of specified DSL statements can be determined according to actual needs.

[0056] After screening the second DSL statement, the historical data to which each second DSL statement belongs can be obtained, and the historical data is determined as the first candidate example.

[0057] S140, using the historical DSL statement in the first candidate example and the historical SQL statement, the quality of the first candidate example is checked to determine the second candidate example.

[0058] The reference example is mainly used to provide examples for the large language model, so that the large language model can accurately learn the process from the natural language query to the DSL statement and the SQL statement, and use the learned content to convert the target query into an accurate target SQL statement.

[0059] To ensure the content in the reference example is accurate, the first candidate example can be subjected to a quality inspection process. The quality inspection process can check the conversion process between the DSL statement and the SQL statement in the first candidate example. Through the quality inspection process, the quality of the first candidate example can be screened to filter out a second candidate example with high quality. Please refer to Figure 3 FIG. 3 shows a flowchart of the quality inspection process.

[0060] As an implementation, when the first candidate example is subjected to the quality inspection process to determine the second candidate example, the historical DSL statement in the first candidate example can be converted into a predicted SQL statement for forward checking, and the historical SQL statement in the first candidate example can be converted into a predicted DSL statement for reverse checking. The first candidate example that passes the forward checking and the reverse checking can be determined as the second candidate example.

[0061] The quality inspection process can include forward checking and reverse checking. The forward checking refers to the checking from the historical DSL statement to the historical SQL statement, and the reverse checking refers to the checking from the historical SQL statement to the historical DSL. The bidirectional checking can improve the reliability of the quality detection, and ensure that a high-quality reference example can be screened out subsequently.

[0062] For each first candidate example, the historical DSL statement therein can be converted into a predicted SQL statement, and the forward checking can be performed based on the predicted SQL statement. Similarly, the historical SQL statement therein can be converted into a predicted DSL statement, and the reverse checking can be performed based on the predicted DSL statement.

[0063] Optionally, when the forward checking is performed, the predicted SQL statement corresponding to the historical DSL statement in the first candidate example can be generated. The key parameters including the number of returned rows, field values, and aggregation values can be extracted from the predicted execution result obtained by executing the predicted SQL statement to obtain predicted parameters. The key parameters can be extracted from the historical execution result corresponding to the historical DSL statement to obtain historical parameters. If the predicted parameters are consistent with the historical parameters, it is determined that the first candidate example passes the forward checking.

[0064] For each first candidate example, a corresponding historical DSL statement in the first candidate example is obtained, an SQL statement corresponding to the historical DSL statement is generated by an existing scheme, and is recorded as a predicted SQL statement. In order to clearly distinguish between the predicted SQL statement and the historical SQL statement corresponding to the original historical SDL statement, the predicted SQL statement can be executed to obtain a predicted execution result, and then the predicted execution result is compared with a historical execution result corresponding to the historical SQL statement. Specifically, key parameters can be extracted from the predicted execution result and recorded as predicted parameters, and key parameters can be extracted from the historical execution result and recorded as historical parameters, and then the predicted parameters and the historical parameters are compared to determine whether they are consistent.

[0065] The key parameters can be set according to actual needs, and in the embodiment of the present application, the key parameters can include the number of returned rows, field values and aggregation values. Each key parameter in the predicted parameters is compared with the historical parameters to determine whether they are consistent; if they are not consistent, it is determined that the first candidate example does not pass the forward checking process; if they are consistent, it is determined that the first candidate example passes the forward checking process.

[0066] In the reverse checking process, the historical SQL statement in the first candidate example can be used to generate a predicted DSL statement corresponding to the historical SQL statement; the overlap ratio of the continuous word sequence between the predicted DSL statement and the historical DSL statement corresponding to the historical SQL statement is calculated; if the overlap ratio is not less than a specified ratio, and the predicted DSL statement is consistent with the historical DSL statement, it is determined that the first candidate example passes the reverse checking process.

[0067] For each historical SQL statement in the first candidate example, the corresponding DSL statement can be inferred by using existing technology, which is recorded as a predicted DSL statement. The overlap ratio of the continuous word sequence between the predicted DSL statement and the historical DSL statement is calculated. The continuous word sequence refers to a sequence composed of n basic units in the text, which can be recorded as n-gram, and the basic unit is usually a word, a character or a character. For example, 1-gram is a sequence composed of 1 basic unit, and 3-gram is a sequence composed of 3 basic units.

[0068] Specifically, when calculating the overlap ratio of the continuous word sequence between the predicted DSL statement and the historical DSL statement, the length of the continuous word sequence can be preset to obtain a plurality of word sequences; for each word sequence, the number of overlaps between the predicted DSL statement and the historical DSL statement is obtained; the number of overlaps is divided by the total number of word sequences in the predicted DSL statement to obtain the proportion of the word sequence; and the geometric mean of the proportions of all word sequences is calculated to obtain the overlap ratio.

[0069] The specified proportion is preset, the overlap proportion is compared with the specified proportion, and a size relationship is obtained. Then, whether the text of the predicted DSL statement and the historical DSL statement is completely consistent can be directly compared. If the overlap proportion is not less than the specified proportion, and the text of the predicted DSL statement and the historical DSL statement is completely consistent, it is determined that the first candidate example passes the reverse verification processing; if the overlap proportion is less than the specified proportion, or the text of the predicted DSL statement and the historical DSL statement is not consistent, it is determined that the first candidate example does not pass the reverse verification processing.

[0070] The first candidate example that passes the forward verification processing and the reverse verification processing is determined as a second candidate example.

[0071] As an implementation manner, in order to reduce the data processing amount, the first candidate example can be subjected to the forward verification processing first, the first candidate example that passes the forward verification processing is determined as an intermediate candidate example, and the intermediate candidate example is subjected to the reverse verification processing, and the intermediate candidate example that passes the reverse verification processing is determined as the second candidate example.

[0072] In this way, a part of the first candidate examples can be filtered out through the forward verification processing first, so that the number of examples that need to be subjected to the reverse verification processing is reduced, so as to reduce the data processing amount. Similarly, the reverse verification processing can be performed first according to actual needs, and then the forward verification processing is performed, which is not limited here.

[0073] S150, based on the weight of the second candidate example and the historical execution result, the second candidate example is subjected to sorting and expansion processing, and a reference example corresponding to the target query is obtained.

[0074] The obtained second candidate examples are examples that pass the quality verification processing, and the reliability of these examples is high. When the large language model is provided, the large language model can be ensured to learn accurate content. These second candidate examples are usually randomly arranged, and have not been subjected to scene coverage detection, which is easy to cause the accuracy of subsequent conversion to be low. In order to further ensure that the reference example can improve the accuracy of the subsequent generated SQL statement, the weight of the second candidate example can be calculated, the second candidate example is sorted according to the weight, and the historical execution result is used to ensure that the example covers multiple scenes.

[0075] As an implementation manner, when the second candidate example is subjected to sorting and expansion processing to obtain a reference example corresponding to the target query, the example weight of the second candidate example can be used to sort and mark the second candidate example, and an example sequence is obtained; and the example sequence is subjected to expansion update processing according to the historical execution result in all second candidate examples, so as to obtain the reference example.

[0076] For each second candidate example, the example weight corresponding to the second candidate example can be calculated, and the second candidate example can be sorted and labeled according to the example weight to obtain an example sequence, so as to avoid random sorting and reduce the learning efficiency of the model. Optionally, when the example sequence is obtained, the similarity between the historical DSL statement in each second candidate example and the intermediate query can be obtained; the example weight corresponding to each second candidate example can be calculated by using the similarity and the number of the second candidate examples; the second candidate example can be updated according to the specified format and the example weight; and the second candidate example can be sorted according to the example weight to obtain the example sequence.

[0077] The similarity between the intermediate query and each historical DSL statement has been calculated in the foregoing, and the similarity corresponding to the historical DSL statement in the second candidate example can be directly obtained. The similarity is renormalized by temperature scaling to obtain the example weight of each second candidate example. Specifically, the example weight of the second candidate example can be calculated by the following formula:

[0078] ;

[0079] wherein, w represents the example weight of the second candidate example; m represents the number of the second candidate examples; sim represents the similarity between the historical DSL statement in the second candidate example and the intermediate query; and T represents the temperature parameter. The similarity between the intermediate query and each historical DSL statement has been calculated in the foregoing, and the similarity corresponding to the historical DSL statement in the second candidate example can be directly obtained. The similarity is renormalized by temperature scaling to obtain the example weight of each second candidate example. Specifically, the example weight of the second candidate example can be calculated by the following formula:

[0080] After the example weight corresponding to each second candidate example is calculated, the content in each second candidate example can be organized according to the specified format, and the example weight can be explicitly labeled in the second candidate example. The specified format can be a chain format of the example weight, the historical query, the historical DSL statement and the historical SQL statement, and the specific historical execution result can not be retained.

[0081] ​​For example, < weight = 0.425 >; NL: count the number of sales department employees in 2023; DSL: Filter (employee, department = sales department & hire_year = 2023) -> Agg (COUNT (employee_id), group_by = department); SQL: SELECT department, COUNT (employee_id) FROM employee WHERE department ='sales department' AND hire_year = 2023 GROUP BY department.

[0082] The example weight corresponding to the second candidate example is added to the second candidate example, and the content in it is adjusted according to the specified format, so as to realize the update of the content contained in the second candidate example.

[0083] The plurality of updated second candidate examples are rearranged in the order of example weight from high to low to obtain an example sequence, so as to ensure that the large language model can preferentially learn examples with higher weights, i.e. higher cosine similarity, so as to improve the utilization efficiency of the model for high-quality examples. For example, the second candidate examples are E1, E2 and E3, and the example sequence obtained by arranging the second candidate examples in the order of example weight from large to small is [E2, E1, E3].

[0084] The obtained example sequence contains a series of high-quality historical data arranged in order. In order to ensure the diversity of reference examples, the example sequence can be updated and expanded based on the historical execution results in the second candidate example to obtain the final reference example.

[0085] Optionally, when the example sequence is updated and expanded based on the historical execution results in the second candidate example to obtain the final reference example, the result type can be determined according to the number of returned rows in the historical execution results corresponding to the second candidate example; the aggregation type can be determined according to the historical DSL statement in the second candidate example; the expansion type is determined by comparing the result type with the preset result type and comparing the aggregation type with the preset aggregation type; the to-be-expanded example is determined from other historical data according to the expansion type, the other historical data being historical data other than the second candidate example; the to-be-expanded example that passes the quality check process is added to the example sequence to obtain the reference example.

[0086] The historical execution result corresponding to each second candidate example is obtained, and the result type of the second candidate example is determined based on the number of returned rows in the historical execution result. A first mapping relationship between a row number range and a preset result type can be set in advance, for example, when the row number range is 0, the corresponding preset result type is "empty result"; when the row number range is [1, 10], the corresponding preset result type is "small result set"; and when the row number range is (10, +∞), the corresponding preset result type is "large result set".

[0087] Therefore, based on the row number range in which the number of returned rows in the historical execution result is located and the first mapping relationship, the preset result type corresponding to the row number range can be determined as the result type of the second candidate example.

[0088] Based on the historical DSL statement, the aggregation type of the second candidate example can be determined. Similarly, a mapping relationship between a preset aggregation type and an aggregation function can be defined in advance. For example, when the aggregation function is COUNT or SUM, the corresponding preset aggregation type is ordinary aggregation; and when the aggregation function is MAX or MIN, the corresponding preset aggregation type is extreme value aggregation.

[0089] Therefore, the aggregation function can be extracted from the historical DSL statement, and based on the second mapping relationship, the corresponding preset aggregation type is determined as the aggregation type of the second candidate example. It can be understood that when the historical DSL statement contains multiple different aggregation functions, the aggregation type of the second candidate example can also have multiple.

[0090] The extension type refers to the preset result type and the preset aggregation type that are not covered by the second candidate example. When determining the extension type, the preset result type other than the result type can be compared with the result type, and the other type in the preset result type is taken as the extension type. Similarly, the aggregation type is compared with the preset aggregation type, and the other type in the preset aggregation type other than the aggregation type is taken as the extension type.

[0091] For example, the result type and the aggregation type of the second candidate example can refer to Table 1.

[0092] Table 1

[0093]

[0094] In the foregoing manner, the expansion type can be determined as empty result, large result set, and extreme value aggregation. The historical data matching the expansion type is searched in the historical data. In the screening, for each expansion type, the historical data matching the expansion type is first found, and then the quality check processing is performed on the historical data in the order of the similarity from large to small between the historical DSL statement corresponding to the historical data and the intermediate query, until the historical data passing the quality check processing is detected. The historical data is updated according to the foregoing example weight and specified format to obtain the expansion example corresponding to the expansion type.

[0095] For example, when the expansion type is empty result, the determined expansion example is E5; when the expansion type is large result set, the determined expansion example is E6; and when the expansion type is extreme value aggregation, the determined expansion example is E7. The final example sequence is [E2, E1, E3, E5, E6, E7].

[0096] The example sequence can be provided to the large language model, so that the large language model generates the target SQL statement corresponding to the target query according to the example sequence. As an implementation, in generating the target SQL statement, the reference example can be analyzed to extract a first generation logic and a second generation logic. The first generation logic is a generation logic of the historical query to the historical DSL statement, and the second generation logic is a generation logic of the historical DSL statement to the historical SQL statement. The target query is converted into a target DSL statement according to the first generation logic, and the target DSL statement is converted into the target SQL statement according to the second generation logic.

[0097] After obtaining the reference example, the reference example can be input to the large language model as part of the prompt word. The large language model can analyze the reference example and learn the first generation logic and the second generation logic therefrom. The first generation logic is a logic learned by the large language model from the reference example for converting the historical query into the historical DSL statement, and the second generation logic is a logic learned by the large language model from the reference example for converting the historical DSL statement into the historical SQL statement.

[0098] The reference example includes multiple samples, each of which includes a historical query, a historical DSL statement, and a historical SQL statement. The large language model can perform logical analysis on the historical query and the corresponding historical DSL statement in each sample to learn a first generation logic. Similarly, the large language model can perform logical analysis on the historical DSL statement and the corresponding historical SQL statement in each sample to learn a second generation logic. The large language model can convert the target query into a corresponding target DSL statement using the learned first generation logic, and then convert the target DSL statement into a corresponding target SQL statement using the learned second generation logic. When determining the reference example, the similarity is first used for preliminary screening, and then the quality of the example is checked to filter out high-quality second candidate examples. The examples are sorted and their covered scenarios are expanded, so that the final reference example is of high quality and comprehensive. Furthermore, when guiding the large language model to generate the target SQL statement using the reference example, the accuracy is higher.

[0099] After generating the target SQL statement, the target SQL statement can be executed to obtain a corresponding execution result, and the execution result is checked and verified. If the execution result matches the intention of the target query, the execution result can be directly displayed to the user, or a target reply corresponding to the target query can be generated based on the execution result, and then the target reply is displayed to the user.

[0100] The scheme provided by the embodiments of the present application can be applied in various scenarios. For example, in the data analysis in the field of industrial manufacturing, the reference example corresponding to the natural language provided by the user can be accurately determined, so that the reference example is used to accurately generate the SQL statement corresponding to the natural language, and then the corresponding data is queried based on the SQL statement to perform intelligent data analysis.

[0101] The method provided by the embodiments of the present application can determine the first candidate example with similar semantics by calculating the cosine similarity between the intermediate query and the historical DSL statement, and then check the mutual conversion between the historical DSL and the historical SQL statement in the first candidate example to filter out the second candidate example with higher quality. The second candidate example is sorted by calculating the weight of the second candidate example, so that the large language model can preferentially learn the high-similarity example to improve the learning efficiency. Finally, the historical execution results are classified, and examples that do not appear in the classification are supplemented to expand the coverage of the examples, and the comprehensiveness of the reference example is ensured. Thus, the reference example with high relevance and high quality can be obtained, and the accuracy of generating the target SQL statement can be effectively improved.

[0102] To better implement the above method, the embodiment of the present application further provides a reference example determination device based on DSL statement, which can be integrated in an electronic device, which can be a terminal, a server, etc. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, a personal computer, etc. The server can be a single server or a server cluster composed of multiple servers. In some embodiments, the reference example determination device based on DSL statement can be integrated in a terminal.

[0103] For example, in the embodiment, the reference example determination device based on DSL statement is integrated in a server.

[0104] For example, as shown in FIG. 2, the reference example determination device 200 based on DSL statement can include an acquisition module 210, an analysis module 220, a candidate module 230, a verification module 240, and a reference module 250. Figure 4 The acquisition module 210 is configured to acquire a target query and a plurality of historical data, wherein the historical data includes historical query statements, historical DSL statements, historical SQL statements, and historical execution results having an association relationship.

[0105] The analysis module 220 is configured to analyze semantics and operation types of the target query to obtain an intermediate query corresponding to the target query.

[0106] The candidate module 230 is configured to determine a first candidate example from the plurality of historical data by using the intermediate query and the historical DSL statements.

[0107] The verification module 240 is configured to perform quality verification processing on the first candidate example by using the historical DSL statement and the historical SQL statement in the first candidate example to determine a second candidate example.

[0108] The reference module 250 is configured to sort and expand the second candidate example based on an example weight of the second candidate example and a historical execution result to obtain a reference example corresponding to the target query.

[0109] In some embodiments, the candidate module 230 is specifically configured to:

[0110] For each historical DSL statement, a cosine similarity between the historical DSL statement and the intermediate query is calculated.

[0111] Based on the cosine similarity, a preset number of historical DSL statements are selected as first DSL statements.

[0112]

[0113] ​determine a second DSL statement in combination with the cosine similarity and an operation type in the first DSL statement;

[0114] determine a history data to which the second DSL statement belongs as a first candidate example.

[0115] In some embodiments, the verification module 240 is specifically configured to:

[0116] convert a history DSL statement in the first candidate example into a predicted SQL statement for forward verification processing;

[0117] convert a history SQL statement in the first candidate example into a predicted DSL statement for reverse verification processing;

[0118] determine a second candidate example from the first candidate example that passes the forward verification processing and the reverse verification processing.

[0119] In some embodiments, the verification module 240 is specifically configured to:

[0120] generate a predicted SQL statement corresponding to a history DSL statement in the first candidate example;

[0121] extract a key parameter from a predicted execution result obtained by executing the predicted SQL statement, to obtain a predicted parameter, the key parameter including a number of returned rows, a field value, and an aggregated value;

[0122] extract the key parameter from a history execution result corresponding to the history DSL statement, to obtain a history parameter;

[0123] if the predicted parameter is consistent with the history parameter, determine that the first candidate example passes the forward verification processing.

[0124] In some embodiments, the verification module 240 is specifically configured to:

[0125] generate a predicted DSL statement corresponding to a history SQL statement in the first candidate example;

[0126] calculate an overlap ratio of continuous word sequences between the predicted DSL statement and a history DSL statement corresponding to the history SQL statement;

[0127] if the overlap ratio is not less than a specified ratio and the predicted DSL statement is consistent with the history DSL statement, determine that the first candidate example passes the reverse verification processing.

[0128] In some embodiments, the reference module 250 is specifically configured to:

[0129] performing a sorting and labeling process on the second candidate examples based on the example weights of the second candidate examples, to obtain an example sequence;

[0130] performing an extension update process on the example sequence according to the historical execution results in all the second candidate examples, to obtain a reference example.

[0131] In some embodiments, the reference module 250 is specifically configured to:

[0132] obtain a similarity between the historical DSL statement in each second candidate example and the intermediate query;

[0133] calculate an example weight corresponding to each second candidate example by using the similarity and the number of the second candidate examples;

[0134] update the second candidate examples according to the specified format and the example weights;

[0135] sort the second candidate examples according to the example weights, to obtain an example sequence.

[0136] In some embodiments, the reference module 250 is specifically configured to:

[0137] determine a result type according to the number of returned rows in the historical execution result corresponding to the second candidate example;

[0138] determine an aggregation type according to the historical DSL statement in the second candidate example;

[0139] determine an extension type by comparing the result type with a preset result type, and comparing the aggregation type with a preset aggregation type;

[0140] determine a to-be-extended example from other historical data according to the extension type, the other historical data being historical data other than the second candidate examples;

[0141] add the to-be-extended example that passes the quality check process to the example sequence, to obtain a reference example.

[0142] In some embodiments, the reference example determination apparatus 200 based on a DSL statement can further include an example use module, which is specifically configured to:

[0143] perform an analysis process on the reference example, to extract a first generation logic and a second generation logic, the first generation logic being a generation logic from a historical query to a historical DSL statement, and the second generation logic being a generation logic from a historical DSL statement to a historical SQL statement.

[0144] Based on the first generation logic, the target query is converted into a target DSL statement;

[0145] According to the second generation logic, the target DSL statement is converted into a target SQL statement.

[0146] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of each unit, please refer to the previous method implementation examples, which will not be repeated here.

[0147] As can be seen from the above, the reference example determination device based on DSL statements in this embodiment can analyze the semantics and operation type of the target query and generate intermediate queries with clearer semantics; using the intermediate queries and historical DSL statements, a first candidate example is initially determined from historical data; then, using the historical DSL statements and historical SQL statements of the first candidate example, the quality verification process of the first candidate example is performed to determine a high-quality second candidate example; then, based on the example weight and historical execution results of the second candidate example, the second candidate example is sorted and expanded, which can ensure the arrangement order and scene coverage of the reference examples, thereby effectively improving the quality of the determined reference examples.

[0148] This invention also provides an electronic device, which can be a terminal, a server, or other similar devices. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0149] In some embodiments, the DSL-based reference example determination device may also be integrated into multiple electronic devices. For example, the DSL-based reference example determination device may be integrated into multiple servers, and the DSL-based reference example determination method of the present invention may be implemented by multiple servers.

[0150] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically:

[0151] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will understand that... Figure 5The electronic device structure shown in the figures is not intended to limit the electronic device, which can include more or fewer components than shown, or combine some components, or arrange the components differently. Among them:

[0152] The processor 310 is the control center of the electronic device, which connects various parts of the entire electronic device through various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 320, and calling data stored in the memory 320. In some embodiments, the processor 310 can include one or more processing cores; in some embodiments, the processor 310 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 310.

[0153] The memory 320 can be used to store software programs and modules, and the processor 310 executes various function applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 320 can also include a memory controller to provide access for the processor 310 to the memory 320.

[0154] The electronic device also includes a power supply 330 for supplying power to various components, and in some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 330 can also include one or more direct current or alternating current power supplies, a recharging system, a power supply failure detection circuit, a power supply converter or inverter, a power supply state indicator, and any other components.

[0155] The electronic device can also include an input module 340, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0156] The electronic device can also include a communication module 350, which in some embodiments can include a wireless module through which the electronic device can perform short-range wireless transmissions, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 can be used to help the user send and receive emails, browse web pages, and access streaming media, etc.

[0157] Although not shown, the electronic device can also include a display unit, etc., which will not be described here. In particular, in the present embodiment, the processor 310 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 320 according to the following instructions, and run the application program stored in the memory 320 by the processor 310, thereby implementing the steps in the method of each embodiment of the application.

[0158] The specific implementation of each of the above operations can refer to the previous embodiments, which will not be described here.

[0159] As can be seen from the above, the electronic device provided by the application can analyze the semantics and operation type of the target query, generate an intermediate query with clearer semantics, use the intermediate query and historical DSL statements to preliminarily determine a first candidate example from historical data, use the historical DSL statements and historical SQL statements of the first candidate example to perform quality checking processing on the first candidate example, determine a second candidate example with high quality, and perform sorting and expansion processing on the second candidate example based on the example weight and historical execution result of the second candidate example, which can ensure the arrangement order and scene coverage of the reference examples, and further effectively improve the quality of the determined reference examples.

[0160] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0161] To this end, the embodiment of the application provides a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the steps in any of the reference example determination methods based on DSL statements provided by the embodiments of the application.

[0162] The storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0163] According to an aspect of the present application, a computer program product or computer program is provided, which comprises computer programs / instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer programs / instructions from the computer readable storage medium, and the processor executes the computer programs / instructions, so that the electronic device executes the method provided in the various optional implementations of the reference example determination aspect or the SQL statement generation aspect provided in the above embodiments.

[0164] Due to the instructions stored in the storage medium, the steps of any of the reference example determination methods based on DSL statements provided in the embodiments of the present application can be executed, thus the beneficial effects of any of the reference example determination methods based on DSL statements provided in the embodiments of the present application can be achieved, which are described in detail in the above embodiments and will not be repeated here.

[0165] The above describes in detail a reference example determination method based on DSL statements and an electronic device provided in the embodiments of the present application. The principles and implementation manners of the present application are described by applying specific examples. The above embodiment descriptions are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for determining a reference example based on a DSL sentence, characterized by, The method comprises: acquiring a target query and a plurality of historical data, the historical data comprising historical query statements, historical DSL statements, historical SQL statements and historical execution results having an association relationship; analyzing semantics and operation types of the target query to obtain an intermediate query corresponding to the target query; determining a first candidate example from the plurality of historical data by using the intermediate query and the historical DSL statements, comprising: calculating, for each of the historical DSL statements, a cosine similarity between the historical DSL statement and the intermediate query; selecting, based on the cosine similarity, a preset number of historical DSL statements as first DSL statements; determining a second DSL statement in combination with the cosine similarity and operation types in the first DSL statements; and determining, as the first candidate example, historical data to which the second DSL statement belongs; performing quality checking processing on the first candidate example by using historical DSL statements and historical SQL statements in the first candidate example to determine a second candidate example, comprising: converting the historical DSL statements in the first candidate example into predicted SQL statements to perform forward checking processing; converting the historical SQL statements in the first candidate example into predicted DSL statements to perform reverse checking processing; and determining, as the second candidate example, the first candidate example that passes the forward checking processing and the reverse checking processing; sorting and expanding the second candidate example based on an example weight of the second candidate example and historical execution results to obtain a reference example corresponding to the target query, comprising: sorting and labeling the second candidate example based on the example weight of the second candidate example to obtain an example sequence; and performing expansion and update processing on the example sequence based on historical execution results in all second candidate examples to obtain a reference example.

2. The method of claim 1, wherein, The converting of the historical DSL statements in the first candidate example into predicted SQL statements to perform forward checking processing comprises: generating, for the historical DSL statements in the first candidate example, predicted SQL statements corresponding to the historical DSL statements; extracting key parameters from predicted execution results obtained by executing the predicted SQL statements to obtain predicted parameters, the key parameters comprising a number of returned rows, field values and aggregation values; extracting the key parameters from historical execution results corresponding to the historical DSL statements to obtain historical parameters; if the predicted parameters are consistent with the historical parameters, determining that the first candidate example passes the forward checking processing.

3. The method of claim 1, wherein, The converting of the historical SQL statements in the first candidate example into predicted DSL statements to perform reverse checking processing comprises: generating, for the historical SQL statements in the first candidate example, predicted DSL statements corresponding to the historical SQL statements; calculating an overlap ratio of continuous word sequences between the predicted DSL statements and historical DSL statements corresponding to the historical SQL statements; If the overlap ratio is not less than a specified ratio and the predicted DSL statement is consistent with the historical DSL statement, it is determined that the first candidate example passes the reverse verification process.

4. The method of claim 1, wherein, The second candidate examples are sorted and labeled based on the example weights of the second candidate examples to obtain an example sequence. The similarity between the historical DSL statement in each second candidate example and the intermediate query is obtained. The example weight corresponding to each second candidate example is calculated using the similarity and the number of second candidate examples. The second candidate examples are updated according to a specified format and the example weights. The second candidate examples are sorted according to the example weights to obtain an example sequence.

5. The method of claim 1, wherein, The example sequence is updated based on the historical execution results in all second candidate examples to obtain a reference example, including: The result type is determined according to the number of returned rows in the historical execution result corresponding to the second candidate example. The aggregation type is determined according to the historical DSL statement in the second candidate example. The expansion type is determined by comparing the result type with a preset result type and comparing the aggregation type with a preset aggregation type. The to-be-expanded example is determined from other historical data according to the expansion type, the other historical data being historical data other than the second candidate examples. The to-be-expanded example that passes the quality verification process is added to the example sequence to obtain a reference example.

6. The method according to any one of claims 1 to 5, characterized in that, After the sorting and expansion of the second candidate examples based on the example weights and historical execution results of the second candidate examples, the method further includes: The reference example is analyzed to extract a first generation logic and a second generation logic, the first generation logic being the generation logic from a historical query to a historical DSL statement, and the second generation logic being the generation logic from a historical DSL statement to a historical SQL statement. The target query is converted into a target DSL statement according to the first generation logic. The target DSL statement is converted into a target SQL statement according to the second generation logic.

7. An electronic device, comprising: A processor and a memory are included, and the memory stores a plurality of instructions; the processor loads the instructions from the memory to execute the steps in the reference example determination method based on DSL statements according to any one of claims 1-6.

Images