Data processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a historical structured query statement; generating a first hash value based on the historical structured query statement, and querying a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value; based on the historical structured query statement, performing parameter replacement on the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement; the example data is used for assisting the language model to output query results. By adopting the method, the generation efficiency of the example data can be improved, so that the output accuracy of the large language model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rise of deep learning technology, when applying large language models for Text2SQL conversion, although they perform well in understanding and generating natural language, they also face the "hallucination" problem, that is, when there is insufficient context information, the model may generate inaccurate SQL (Structured Query Language) statements that do not match the actual query intent. This can be effectively alleviated by using Few-shot Prompting sample data.

[0003] In traditional technology, the performance of a model on a specific task is improved by providing it with a small amount of highly relevant example data. In order to provide the model with Few-shot Prompting example data that is closely related to a specific query task, it is usually necessary to manually create example data based on each specific query task, and manually update and maintain the example data.

[0004] However, in traditional technologies, sample data is manually created, which has high operational complexity and low efficiency in generating examples and cannot meet the application requirements of large language models. Summary of the invention

[0005] Based on this, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0006] In a first aspect, the present application provides a data processing method, comprising:

[0007] Get historical structured query statements;

[0008] Generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value;

[0009] Based on the historical structured query statement, parameters are replaced on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

[0010] In one embodiment, the method further comprises:

[0011] The example data of each of the historical structured query statements is stored in a database instance to construct an example data set.

[0012] In one embodiment, the method further comprises:

[0013] Receiving a query request from a user, wherein the query request includes query information in text format;

[0014] Based on the query information, matching target example data associated with the query information in the pre-constructed example data set;

[0015] Inputting the target example data into the language model to generate a target structured query statement corresponding to the query information;

[0016] Based on the language model and the target structured query statement, a query result corresponding to the query request is output.

[0017] In one embodiment, generating a first hash value based on the historical structured query statement includes:

[0018] Parameterizing the historical structured query statement to obtain a first structured query statement;

[0019] The first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized to generate a first hash value corresponding to the first structured query statement.

[0020] In one embodiment, there is a mapping relationship between the structured query statement template and the natural language description template, and the querying of the target structured query statement template and the target natural language description template matching the historical structured query statement according to the first hash value includes:

[0021] Searching for a target hash value consistent with the first hash value among the hash values ​​corresponding to the structured query statement template;

[0022] Determine the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical structured query statement;

[0023] Based on the mapping relationship between the structured query statement template and the natural language description template, a target natural language description template corresponding to the target structured query statement template is determined.

[0024] In one embodiment, before querying the target structured query statement template and the target natural language description template matching the historical structured query statement according to the first Hash value, the method further includes:

[0025] According to the first Hash value, querying among the Hash values ​​corresponding to the created example data set whether there is a target Hash value that is the same as the first Hash value;

[0026] If the target hash value exists, the example data corresponding to the target hash value is determined as the example data of the historical structured query statement.

[0027] In a second aspect, the present application further provides a data processing device, comprising:

[0028] The acquisition module is used to obtain historical structured query statements;

[0029] A matching module, configured to generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value;

[0030] The first generation module is used to replace parameters of the target structured query statement template and the target natural language description template based on the historical structured query statement to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Get historical structured query statements;

[0033] Generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value;

[0034] Based on the historical structured query statement, parameters are replaced on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

[0035] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0036] Get historical structured query statements;

[0037] Generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value;

[0038] Based on the historical structured query statement, parameters are replaced on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

[0039] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0040] Get historical structured query statements;

[0041] Generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value;

[0042] Based on the historical structured query statement, parameters are replaced on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

[0043] The above-mentioned data processing method, device, computer equipment, computer-readable storage medium and computer program product obtain a historical structured query statement; generate a first hash value based on the historical structured query statement, and query the target structured query statement template and the target natural language description template that match the historical structured query statement according to the first hash value; based on the historical structured query statement, perform parameter replacement on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting query results. This method is used to generate sample data of the historical structured query statement based on the pre-built structured query statement template, natural language description template and the historical structured query statement, thereby avoiding the manual creation of sample data, simplifying the difficulty of generating sample data, and improving the generation efficiency of sample data. Furthermore, the sample data of the historical structured query statement is used as the input prompt of the large language model to assist the large language model in outputting results and improve the output accuracy of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 is a flow chart of a data processing method in one embodiment;

[0046] Figure 2 A schematic diagram of a process for constructing an example data set in one embodiment;

[0047] Figure 3 A schematic diagram of a flow chart of implementing a user query step based on a language model in one embodiment;

[0048] Figure 4 A schematic diagram of a process for generating a first hash value of a historical SQL statement in one embodiment;

[0049] Figure 5 A flowchart of the steps of querying and obtaining a target structured query statement template and a target natural language description template corresponding to a historical SQL statement in one embodiment;

[0050] Figure 6 A flowchart of example data steps for verifying whether a historical SQL statement already exists in an example data set in one embodiment;

[0051] Figure 7 is a structural block diagram of a data processing device in one embodiment;

[0052] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In one embodiment, in order to avoid the high complexity and low efficiency of manually creating sample data, Figure 1As shown, a data processing method is provided, in which the purpose is to generate corresponding sample data for all successfully executed SQL statements in the database, and then, when the language model requires auxiliary input of sample data, select target sample data from the sample data generated by the SQL statement and input it into the language model. This embodiment uses the method applied to the terminal as an example for illustration, and it can be understood that the method can also be applied to the server, and can also be applied to the system including the terminal and the server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0055] Step 102: Obtain historical structured query statements.

[0056] In implementation, the terminal obtains historical structured query statements (SQL statements) from the database instance. The historical SQL statements are SQL statements that have been successfully executed in the database instance. In order to generate corresponding sample data based on the SQL statements, the historical SQL statements are first parameterized, that is, the table name table, field name field and constant value in the historical SQL statements are converted into common tags, for example, a historical SQL is converted into ${table_a}, ${field_1}, ${value_1}.

[0057] Step 104: Generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value.

[0058] In the implementation, a series of structured query statement templates (i.e., SQL templates) and natural language description templates are pre-defined in the terminal, and mapping rules between structured query statement templates and natural language description templates are constructed. For example, a SQL template A is constructed: SELECT ${field_2} FROM ${table_1} WHERE ${field_1} = ${value_1}; the natural language description template corresponding to the SQL template A is: get the corresponding ${field_2} from ${table_1} according to the value of ${field_1} being ${value_1}. The natural language description template and the SQL template have been mapped based on a preset programming language. In addition, for complex SQL statements containing subqueries, the complex SQL statements are also described in a nested manner by decomposing them into multiple single SQL statements.

[0059] For example, complex SQL template B: SELECT ${field_4} FROM ${table_2} WHERE ${field_3}= ${subquery};

[0060] The natural language description template corresponding to the complex SQL template B is: ${subquery description}, and the corresponding ${field_4} is obtained from ${table_2} according to ${field_3}. Among them, ${subquery description} can be the above-mentioned SQL template A, that is, it can be in the form of a single SQL statement.

[0061] After generating a series of SQL templates and natural language description templates corresponding to the SQL templates, the generated SQL templates and natural language description templates are stored in the database instance, and the hash values ​​of each sample data, the hash values ​​of the structured query statement template (i.e., the SQL template), and the hash values ​​of the natural language description template are also stored in the database instance. The terminal generates a first hash value for the parameterized historical structured query statement, and queries the database instance according to the first hash value, that is, the first hash value generated by the historical structured statement is compared with the hash values ​​of each sample data for consistency. If there is no hash value consistent with the first hash value in the hash values ​​of each sample data, it indicates that there is no sample data corresponding to the historical SQL statement in the current database instance, and therefore, it is necessary to generate sample data based on the SQL statement. Then, the terminal first matches the hash values ​​of the structured query statement templates stored in the database instance and the hash values ​​of the natural language description template based on the first hash value of the historical SQL statement, and queries the target structured query statement template and the target natural description language template that match the historical structured query statement.

[0062] Step 106: Based on the historical structured query statements, parameters are replaced on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statements.

[0063] The sample data is used to assist the language model in outputting query results.

[0064] In implementation, after determining the target structured query statement template and the target natural language description template corresponding to the historical structured query statement, the terminal performs parameter replacement on the target structured query statement template and the target natural language description template based on the historical structured query statement, that is, replaces the actual parameter values ​​contained in the historical structured query statement into the target structured query statement template and the target natural language description template, and generates example data of the historical structured query statement, and the format of the example data is: (natural language description, historical SQL).

[0065] In the above data processing method, a historical structured query statement is obtained; a first hash value is generated based on the historical structured query statement, and a target structured query statement template and a target natural language description template that match the historical structured query statement are queried according to the first hash value; based on the historical structured query statement, the target structured query statement template and the target natural language description template are replaced with parameters to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting query results. This method is used to generate sample data of historical structured query statements based on pre-built structured query statement templates, natural language description templates and historical structured query statements, thereby avoiding manual creation of sample data, simplifying the difficulty of generating sample data, and improving the efficiency of generating sample data. Furthermore, the sample data of historical structured query statements is used as input prompts for a large language model to assist the large language model in outputting results and improve the output accuracy of the large language model.

[0066] In an exemplary embodiment, after obtaining the sample data corresponding to the historical structured query statement, the terminal may store the sample data corresponding to the historical structured query statement for application of the language model, such as Figure 2 As shown, the method also includes:

[0067] Step 202: Store sample data of each historical structured query statement in a database instance to construct a sample data set.

[0068] In the implementation, the terminal stores the sample data of each historical structured query statement in the database instance to build a sample data set. In the application process of the language model, if the model faces the "hallucination" problem when understanding and generating natural language, it can be effectively alleviated through Few-shot Prompting, which uses a small amount of sample data related to the user's query statement as an input prompt to assist the language model in outputting results.

[0069] In this embodiment, the example data of the historical structured query statements are stored in the database instance. When a user queries, the relevant example data is quickly found in the example data set of the database instance according to the query request input by the user, avoiding the complexity of manually generating example data.

[0070] In an exemplary embodiment, sample data corresponding to each historical SQL statement is stored in a database instance, and a language model is integrated in the terminal. The language model may be a question-answering model based on an artificial intelligence algorithm. In the application process of the language model, Figure 3 As shown, the method also includes:

[0071] Step 302: Receive a query request from a user.

[0072] The query request includes query information in text format.

[0073] In implementation, in the language model integrated in the terminal, a user may initiate a query request, which includes query information in text format. For example, the query request may be a target question in the form of a query, such as "get the names of all employees over 50 years old." The terminal receives the user's query request and runs the language model.

[0074] Step 304 : Based on the query information, match the target example data associated with the query information in the pre-built example data set.

[0075] In implementation, the terminal parses the query request expressed in natural language to obtain the query information in the query request. Then, based on the query information, the terminal preferentially matches the target sample data associated with the query information in the constructed sample data set. Specifically, the database instance of the terminal stores sample data generated by each historical SQL statement. If the current query information corresponds to a query content similar to that of the historical SQL statement, the sample data of the historical SQL statement with high similarity can be directly used as the target sample data and input into the language model for auxiliary output.

[0076] Optionally, if no target sample data matching the query information is found in the pre-built sample data set, target sample data matching the query information is generated according to the processing of steps 104 to 106, and the process of generating the target sample data has been described above and will not be repeated here.

[0077] Step 306: input the target sample data into the language model to generate a target structured query statement corresponding to the query information.

[0078] In implementation, the target sample data includes natural language description and SQL statements, i.e. (natural language description, historical SQL). In the query request "Get the names of all employees over 50 years old" in step 302 above, if the target sample data related to the query information of the query request is "get names from employees with age greater than 30", "SELECT name FROM employees WHERE age > 30". After obtaining the target sample data, the target sample data is used as a language model prompt template. The terminal inputs the target sample data into the language model, learns the language description method and SQL statement rules of the target sample data through the language model, and generates the correct target structured query statement corresponding to the query information input by the user, that is, generates the correct SQL statement corresponding to the query information: SELECT name FROM employees WHERE age > 50.

[0079] Step 308: output the query result corresponding to the query request based on the language model and the target structured query statement.

[0080] In implementation, after obtaining the target SQL statement corresponding to the query information, the terminal analyzes and calculates the target structured query statement (target SQL statement) based on the language model and queries related data, and outputs the query result corresponding to the query request. That is, for the query request of the target question "get the names of all employees over 50 years old" input by the user, a list of the names of all employees over 50 years old in the specific related data set is given as the query result.

[0081] In this embodiment, historical SQL is converted into sample data to form a sample data set. When a user queries, the corresponding sample data can be found according to the query input by the user, and the sample data is added to the prompt as a few-shot to assist the model in generating correct SQL statements, thereby improving the performance and accuracy of the language model.

[0082] In an exemplary embodiment, Figure 4 As shown, the specific processing process of generating the first hash value based on the historical structured query statement in step 104 includes:

[0083] Step 401: parameterize the historical structured query statement to obtain a first structured query statement.

[0084] In implementation, the terminal performs parameterization processing on the historical structured query statement to obtain the first structured query statement. Specifically, the parameter values ​​in the historical structured query statement are replaced with general tags to obtain the first structured query statement. For example, the historical SQL statement is: SELECT name FROM employees WHERE age > 30. For the historical structured query statement, "name", "employees" and "age > 30" are replaced with corresponding general tags: ${table_a}, ${field_1}, ${value_1}, etc. The first structured query statement is obtained: SELECT ${field_2}FROM ${table_1} WHERE ${field_1} = ${value_1}. Among them, the purpose of parameterizing the historical SQL statement into the first structured query statement is to match the structured query statement template in the database instance when the historical SQL does not generate the corresponding sample data.

[0085] Step 402: parse the first structured query statement into an abstract syntax tree, and serialize the abstract syntax tree to generate a first hash value corresponding to the first structured query statement.

[0086] In implementation, the terminal parses the first structured query statement into an abstract syntax tree (ASA), which represents the syntax rules of the programming language in a tree form, uses nodes to represent language constructs, and each node represents a structure in the source code (such as an expression, statement, control flow, etc.). Then, the terminal serializes the abstract syntax tree and generates a first hash value corresponding to the first structured query statement based on the serialized sequence data.

[0087] In this embodiment, the historical SQL statements are parameterized and serialized to generate the first hash value of the historical SQL statements, so as to perform query matching in the example data set of the database instance through the first hash value, thereby improving the efficiency of query matching.

[0088] In an exemplary embodiment, Figure 5 As shown, there is a mapping relationship between the structured query statement template and the natural language description template. The specific processing process of querying the target structured query statement template and the target natural language description template matching the historical structured query statement according to the first hash value in step 104 includes:

[0089] Step 501: Search for a target hash value that is consistent with a first hash value among hash values ​​corresponding to a structured query statement template.

[0090] In implementation, the hash value of the structured query statement template (ie, SQL template) and the hash value of the natural language description template are stored in the database instance. Among the hash values ​​corresponding to the structured query statement template, the terminal searches for a target hash value that is consistent with the first hash value.

[0091] Step 502: determine the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical structured query statement.

[0092] In implementation, the terminal determines the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical SQL statement. Specifically, when there is no corresponding sample data for the historical SQL statement, the corresponding sample data is generated based on the structured query statement template corresponding to the historical SQL statement. Therefore, the terminal first performs a template query based on the first hash value, and queries a target hash value that is consistent with the first hash value. Then, the structured query statement template corresponding to the target hash value is the target structured query statement template that matches the historical SQL statement.

[0093] Step 503: Determine a target natural language description template corresponding to the target structured query statement template based on the mapping relationship between the structured query statement template and the natural language description template.

[0094] In implementation, after determining the structured query statement template, since a mapping relationship is predefined between the structured query statement template and the natural language description template, the terminal can directly determine the target natural language description template corresponding to the target structured query statement template based on the mapping relationship between the structured query statement template and the natural language description template. In this way, the target structured query statement and the target natural language description template required to generate the sample data are determined.

[0095] In this embodiment, the search is completed in a relatively short time by matching the hash value, which greatly improves the query efficiency. At the same time, based on the mapping relationship between the structured query statement template and the natural language description template, the target natural language description template corresponding to the target structured query statement template for generating the sample data is quickly determined, thereby improving the sample data generation efficiency.

[0096] In an exemplary embodiment, during the generation of sample data of historical SQL, before constructing the sample data of historical SQL, it is possible to query in the database instance whether the sample data of the historical SQL already exists. If it exists, it is not necessary to generate it, and the next historical SQL statement to generate sample data is directly processed. Therefore, before generating sample data for the historical SQL statement, the sample data of the historical SQL statement can be checked first. The checking process is still completed with the help of the hash value of the historical SQL statement to improve the data query efficiency, such as Figure 6 As shown, after step 104, the method further includes:

[0097] Step 601: According to the first Hash value, query whether there is a target Hash value that is the same as the first Hash value in each Hash value corresponding to the created example data set.

[0098] In implementation, hash values ​​of each created example data are stored in a database instance. The terminal queries, based on the first hash value of the historical SQL, whether there is a target hash value identical to the first hash value in each hash value corresponding to the created example data set of the database instance. If the target hash value does not exist, it indicates that the current historical SQL statement has no corresponding example data, and it is necessary to generate example data for the historical SQL statement, i.e., executing step 104 of the above embodiment of querying the target structured query statement template and the target natural language description template that match the historical structured query statement based on the first hash value, until step 106, to obtain example data for the historical SQL statement.

[0099] Step 602: If the target hash value exists, the sample data corresponding to the target hash value is determined as the sample data of the historical structured query statement.

[0100] In implementation, if the target hash value exists in the example data set, it indicates that the historical SQL statement has corresponding example data in the example data set. Therefore, there is no need to repeatedly process the historical SQL statement, but the example data corresponding to the target hash value can be directly determined as the example data of the historical structured query statement, that is, the association relationship between the historical SQL and the example data is clarified.

[0101] In this embodiment, data query performed with the help of the first hash value has high query efficiency, and if there is sample data of historical SQL in the database instance, the query is directly performed based on the first hash value of the historical SQL statement, without the need to repeatedly generate sample data, thus saving data processing resources.

[0102] In a specific embodiment, a specific example process of generating sample data based on a data processing method and applying the sample data during a user query is provided. The specific example process is as follows:

[0103] Step 1: SQL templates and natural language description templates are pre-defined and stored in the database instance, wherein there is a mapping relationship between the SQL templates and the natural language description templates, for example:

[0104] SQL template: SELECT ${field_2} FROM ${table_1} WHERE ${field_1} > ${value_1};

[0105] Natural language description template: Get ${field_2} from ${table_1} where the value of ${field_1} is greater than ${value_1}.

[0106] Step 2: Get the historical SQL statements that have been successfully executed and analyze them:

[0107] (1) Take the historical SQL statement: SELECT name FROM employees WHERE age > 30 as an example, parameterize the historical SQL statement and replace it with: SELECT $field_1 FROM $table WHERE $field_2 > $value_1;

[0108] (2) Generate an AST from the parameterized historical SQL statements and serialize them to obtain the first hash value.

[0109] (3) Based on the first hash value, template matching is performed in the database instance, and the matching SQL template is confirmed to be: "SELECT $field FROM $table WHERE $field > $value;", and the natural language description template is: "Get ${field_2} from ${table_1} whose value is greater than ${value_1}." After replacing the natural language description template with the specific value, the sample data is formed: ("Get name from employees whose age is greater than 30", "SELECT nameFROM employees WHERE age > 30").

[0110] (4) Add the sample data of the newly generated historical SQL statements to the sample data set of the database instance and update the sample data set.

[0111] Step 4: User query processing:

[0112] (1) User query input: "Get the names of all employees over 50 years old."

[0113] (2) The system matches similar example data in the example dataset: (natural language, historical SQL), that is, ("get name from employees whose age is greater than 30", "SELECT name FROM employees WHERE age >30").

[0114] (3) This sample data is added to the prompt of the large language model as a few-shot, which helps the model generate the correct SQL statement: SELECT name FROM employees WHERE age > 50.

[0115] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0117] In an exemplary embodiment, Figure 7 As shown, a data processing device 700 is provided, comprising: an acquisition module 701, a matching module 702 and a first generation module 703, wherein:

[0118] The acquisition module 701 is used to acquire historical structured query statements;

[0119] A matching module 702 is used to generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first hash value;

[0120] The first generating module 703 is used to replace parameters of a target structured query statement template and a target natural language description template based on the historical structured query statement to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

[0121] In one embodiment, the data processing device 700 further includes:

[0122] The construction module is used to store the sample data of each historical structured query statement in the database instance to build a sample data set.

[0123] In one embodiment, the data processing device 700 further includes:

[0124] A receiving module, used for receiving a user's query request, wherein the query request includes query information in text format;

[0125] A query matching module, used to match target sample data associated with the query information in a pre-built sample data set based on the query information;

[0126] A second generation module is used to input the target sample data into the language model to generate a target structured query statement corresponding to the query information;

[0127] The output module is used to output the query results corresponding to the query request based on the language model and the target structured query statement.

[0128] In one embodiment, the matching module 702 is specifically configured to perform parameterization processing on the historical structured query statement to obtain a first structured query statement;

[0129] The first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized to generate a first hash value corresponding to the first structured query statement.

[0130] In one embodiment, there is a mapping relationship between the structured query statement template and the natural language description template, and the matching module 702 is specifically used to search for a target hash value consistent with the first hash value among the hash values ​​corresponding to the structured query statement template;

[0131] Determine the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical structured query statement;

[0132] Based on the mapping relationship between the structured query statement template and the natural language description template, a target natural language description template corresponding to the target structured query statement template is determined.

[0133] In one embodiment, the apparatus 700 further includes:

[0134] A query module, configured to query, based on the first hash value, whether there is a target hash value identical to the first hash value in each hash value corresponding to the created example data set;

[0135] The determination module is used to determine the sample data corresponding to the target hash value as the sample data of the historical structured query statement if the target hash value exists.

[0136] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0137] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a data processing method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0138] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0140] Get historical structured query statements;

[0141] Generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first hash value;

[0142] Based on the historical structured query statements, parameters of the target structured query statement template and the target natural language description template are replaced to generate sample data of the historical structured query statements; the sample data is used to assist the language model in outputting the query results.

[0143] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0144] The sample data of each historical structured query statement is stored in the database instance to build a sample data set.

[0145] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0146] Receiving a query request from a user, where the query request includes query information in text format;

[0147] Based on the query information, matching target example data associated with the query information in a pre-built example data set;

[0148] Input the target sample data into the language model to generate a target structured query statement corresponding to the query information;

[0149] Based on the language model and the target structured query statement, the query result corresponding to the query request is output.

[0150] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0151] Parameterizing the historical structured query statement to obtain a first structured query statement;

[0152] The first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized to generate a first hash value corresponding to the first structured query statement.

[0153] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0154] Searching for a target hash value that is consistent with the first hash value among the hash values ​​corresponding to the structured query statement template;

[0155] Determine the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical structured query statement;

[0156] Based on the mapping relationship between the structured query statement template and the natural language description template, a target natural language description template corresponding to the target structured query statement template is determined.

[0157] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0158] According to the first hash value, querying among the hash values ​​corresponding to the created example data set whether there is a target hash value that is the same as the first hash value;

[0159] If the target hash value exists, the sample data corresponding to the target hash value is determined as the sample data of the historical structured query statement.

[0160] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0161] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0162] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0163] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0164] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method comprises: Get historical structured query statements; Generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value; Based on the historical structured query statement, parameters are replaced on the target structured query statement template and the target natural language description template to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

2. The method according to claim 1, characterized in that: The method further comprises: The example data of each of the historical structured query statements is stored in a database instance to construct an example data set.

3. The method according to claim 2, characterized in that The method further comprises: Receiving a query request from a user, wherein the query request includes query information in a text format; Based on the query information, matching target example data associated with the query information in the pre-constructed example data set; Inputting the target example data into the language model to generate a target structured query statement corresponding to the query information; Based on the language model and the target structured query statement, a query result corresponding to the query request is output.

4. The method according to claim 1, characterized in that The generating a first hash value based on the historical structured query statement includes: Parameterizing the historical structured query statement to obtain a first structured query statement; The first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized to generate a first hash value corresponding to the first structured query statement.

5. The method according to claim 1, characterized in that There is a mapping relationship between the structured query statement template and the natural language description template, and the querying of the target structured query statement template and the target natural language description template matching the historical structured query statement according to the first Hash value includes: Searching for a target hash value consistent with the first hash value among the hash values ​​corresponding to the structured query statement template; Determine the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical structured query statement; Based on the mapping relationship between the structured query statement template and the natural language description template, a target natural language description template corresponding to the target structured query statement template is determined.

6. The method according to claim 2, characterized in that Before searching for a target structured query statement template and a target natural language description template matching the historical structured query statement according to the first Hash value, the method further includes: According to the first Hash value, querying among the Hash values ​​corresponding to the created example data set whether there is a target Hash value that is the same as the first Hash value; If the target hash value exists, the example data corresponding to the target hash value is determined as the example data of the historical structured query statement.

7. A data processing device, characterized in that: The device comprises: The acquisition module is used to obtain historical structured query statements; A matching module, configured to generate a first Hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template that match the historical structured query statement according to the first Hash value; The first generation module is used to replace parameters of the target structured query statement template and the target natural language description template based on the historical structured query statement to generate sample data of the historical structured query statement; the sample data is used to assist the language model in outputting the query result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images