Data processing method and device, computer device, readable storage medium and program product

Abstract

The application relates to a data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises: obtaining a historical structured query statement; generating a first hash value based on the historical structured query statement, and querying a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value; performing parameter replacement on the target structured query statement template and the target natural language description template based on the historical structured query statement to generate example data of the historical structured query statement; and the example data is used to assist a language model to output a query result. The method can improve the generation efficiency of the example data, and further improve the output accuracy of the large language model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a data processing method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the rise of deep learning technology, when applying large language models for Text2SQL conversion, although they perform well in understanding and generating natural language, they also face the "hallucination" problem, that is, when there is a lack of sufficient context information, the model may generate inaccurate or inconsistent SQL (Structured Query Language) statements with the actual query intent. At this time, Few-shot Prompting (few-shot prompting) example data can be used to effectively alleviate this problem.

[0003] In traditional technology, a small amount of highly relevant example data is provided to the model to improve its performance on specific tasks. However, in order to provide Few-shot Prompting example data closely related to specific query tasks, it is usually necessary to manually create example data based on each specific query task and manually update and maintain the example data.

[0004] However, in traditional technology, the manual creation of example data is complex and inefficient, and cannot meet the application requirements of large language models. SUMMARY

[0005] Therefore, it is necessary to provide a data processing method, device, computer equipment, computer readable storage medium and computer program product to solve the above technical problems.

[0006] In a first aspect, the present application provides a data processing method, comprising:

[0007] obtaining a historical structured query statement;

[0008] generating a first hash value based on the historical structured query statement, and querying a target structured query statement template and a target natural language description template matching the historical structured query statement according to the first hash value;

[0009] based on the historical structured query statement, performing parameter substitution on the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement; the example data is used to assist the language model to output a query result.

[0010] In one embodiment, the method further comprises:

[0011] Store the example data of each of the historical structured query statements into a database instance to build an example data set.

[0012] In one of the embodiments, the method further comprises:

[0013] receiving a query request of a user, the query request containing query information in a text format;

[0014] based on the query information, matching target example data associated with the query information in the pre-built example data set;

[0015] inputting the target example data into the language model to generate a target structured query statement corresponding to the query information;

[0016] based on the language model and the target structured query statement, outputting a query result corresponding to the query request.

[0017] In one of the embodiments, the generating a first hash value based on the historical structured query statement comprises:

[0018] parameterizing the historical structured query statement to obtain a first structured query statement;

[0019] parsing the first structured query statement into an abstract syntax tree and serializing the abstract syntax tree to generate a first hash value corresponding to the first structured query statement.

[0020] In one of the embodiments, there is a mapping relationship between the structured query statement template and the natural language description template, and the querying, based on the first hash value, of a target structured query statement template and a target natural language description template matched with the historical structured query statement comprises:

[0021] in each of the hash values corresponding to the structured query statement template, querying a target hash value identical to the first hash value;

[0022] determining the structured query statement template corresponding to the target hash value as the target structured query statement template matched with the historical structured query statement;

[0023] based on the mapping relationship between the structured query statement template and the natural language description template, determining a target natural language description template corresponding to the target structured query statement template.

[0024] In one of the embodiments, before the querying, based on the first hash value, of a target structured query statement template and a target natural language description template matched with the historical structured query statement, the method further comprises:

[0025] According to the first hash value, whether there is a target hash value same as the first hash value in each hash value corresponding to the created example data set is queried;

[0026] If the target hash value exists, example data corresponding to the target hash value is determined as the example data of the historical structured query statement.

[0027] In a second aspect, the present application further provides a data processing apparatus, comprising:

[0028] An obtaining module is configured to obtain a historical structured query statement.

[0029] A matching module is configured to generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value.

[0030] A first generating module is configured to perform parameter replacement on the target structured query statement template and the target natural language description template based on the historical structured query statement, to generate example data of the historical structured query statement; the example data is used to assist a language model to output a query result.

[0031] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

[0032] A historical structured query statement is obtained.

[0033] A first hash value is generated based on the historical structured query statement, and a target structured query statement template and a target natural language description template matched with the historical structured query statement are queried according to the first hash value.

[0034] Parameter replacement is performed on the target structured query statement template and the target natural language description template based on the historical structured query statement, to generate example data of the historical structured query statement; the example data is used to assist a language model to output a query result.

[0035] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program realizes the following steps when executed by a processor:

[0036] A historical structured query statement is obtained.

[0037] generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value;

[0038] based on the historical structured query statement, perform parameter replacement on the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement; the example data is used to assist the language model to output a query result.

[0039] In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0040] obtaining a historical structured query statement;

[0041] generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value;

[0042] based on the historical structured query statement, perform parameter replacement on the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement; the example data is used to assist the language model to output a query result.

[0043] The above data processing method, device, computer equipment, computer readable storage medium and computer program product, obtain a historical structured query statement; generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value; based on the historical structured query statement, perform parameter replacement on the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement; the example data is used to assist the language model to output a query result. By using the method, based on the pre-constructed structured query statement template, natural language description template and historical structured query statement, example data of the historical structured query statement is generated, thereby avoiding manual creation of example data, simplifying the difficulty of generating example data, improving the efficiency of generating example data, and then using the example data of the historical structured query statement as an input prompt of the large language model to assist the large language model to output a result, thereby improving the output accuracy of the large language model. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained on the basis of these drawings without any creative effort.

[0045] Figure 1 Flowchart of a data processing method in an embodiment;

[0046] Figure 2 Flowchart of a step of constructing an example data set in an embodiment;

[0047] Figure 3 Flowchart of a step of implementing a user query based on a language model in an embodiment;

[0048] Figure 4 Flowchart of a step of generating a first hash value of a historical SQL statement in an embodiment;

[0049] Figure 5 Flowchart of a step of querying a target structured query statement template and a target natural language description template corresponding to the historical SQL statement in an embodiment;

[0050] Figure 6 Flowchart of a step of checking whether an example data of the historical SQL statement exists in the example data set in an embodiment;

[0051] Figure 7 Block diagram of a data processing apparatus in an embodiment;

[0052] Figure 8 Internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0054] In an embodiment, in order to avoid the complexity of artificially creating example data being too high and the efficiency being low, the example data is constructed by using a language model in the embodiment. Figure 1As shown, a data processing method is provided, in which all successfully executed SQL statements in the database are generated corresponding example data, and then when the language model needs auxiliary input of example data, target example data is selected from the generated example data of the SQL statement and input into the language model. This embodiment illustrates the application of the method to the terminal. It can be understood that the method can also be applied to the server, and can also be applied to a system including the terminal and the server, and is realized through the interaction of the terminal and the server. In this embodiment, the method includes the following steps:

[0055] Step 102, obtaining a historical structured query statement.

[0056] In implementation, the terminal obtains a historical structured query statement (SQL statement) from a database instance, which is a successfully executed SQL statement in the database instance. In order to generate corresponding example data based on the SQL statement, the historical SQL statement is first parameterized, that is, the table name table, the field name field and the constant value in the historical SQL statement are converted into general markers, for example, a certain historical SQL is converted into ${table_a}, ${field_1}, ${value_1}.

[0057] Step 104, generating a first hash value based on the historical structured query statement, and querying a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value.

[0058] In implementation, a series of structured query statement templates (i.e. SQL templates) and natural language description templates are predefined in the terminal, and a mapping rule between the structured query statement template and the natural language description template is constructed. For example, a SQL template A is constructed: SELECT ${field_2} FROM ${table_1} WHERE ${field_1} = ${value_1};The natural language description template corresponding to the SQL template A is: from ${table_1} according to the value of ${field_1} ${field_2} corresponding to ${value_1} is obtained. The natural language description template and the SQL template have a mapping relationship based on the preset program language. Moreover, for complex SQL statements containing subqueries, the complex SQL statements are also described by nesting in the form of multiple single SQL statements.

[0059] For example, a complex SQL template B: SELECT ${field_4} FROM ${table_2} WHERE ${field_3}= ${subquery};

[0060] The natural language description template corresponding to the complex SQL template B is: ${description of the subquery}, and the corresponding ${field 4} is obtained from ${table 2} according to ${field 3}. Wherein, ${description of the subquery} can be the above-mentioned SQL template A, that is, in the form of a single SQL statement.

[0061] After generating a series of SQL templates and natural language description templates corresponding to the SQL templates, the generated SQL templates and natural language description templates are stored in the database instance, and the hash values of each example data, the hash values of the structured query statement templates (i.e. SQL templates) and the hash values of the natural language description templates are also stored in the database instance. The terminal generates a first hash value for the parameterized historical structured query statement, and queries in the database instance according to the first hash value, that is, consistency comparison is performed between the first hash value generated by the historical structured statement and the hash values of each example data. If there is no hash value consistent with the first hash value in the hash values of each example data, it indicates that there is no example data corresponding to the historical SQL statement in the current database instance, therefore, it is necessary to generate example data based on the SQL statement. Then the terminal first performs hash value matching based on the first hash value of the historical SQL statement in the hash values of each structured query statement template and the hash values of the natural language description templates stored in the database instance, and queries the target structured query statement template and the target natural language description template matching the historical structured query statement.

[0062] Step 106, based on the historical structured query statement, parameter replacement is performed on the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement.

[0063] Wherein, the example data is used to assist the language model to output the query result.

[0064] In implementation, after determining the target structured query statement template and the target natural language description template corresponding to the historical structured query statement, the terminal performs parameter replacement on the target structured query statement template and the target natural language description template based on the historical structured query statement, that is, the actual parameter values contained in the historical structured query statement are replaced into the target structured query statement template and the target natural language description template to generate example data of the historical structured query statement, and the form of the example data is: (natural language description, historical SQL).

[0065] In the data processing method, a historical structured query statement is obtained; a first hash value is generated based on the historical structured query statement, and a target structured query statement template and a target natural language description template matching the historical structured query statement are queried according to the first hash value; the target structured query statement template and the target natural language description template are subjected to parameter replacement based on the historical structured query statement, to generate example data of the historical structured query statement; and the example data is used to assist a language model to output a query result. By using the method, example data of a historical structured query statement is generated based on a pre-constructed structured query statement template, a natural language description template and the historical structured query statement, thereby avoiding manual creation of example data, simplifying the difficulty of generating example data, improving the efficiency of generating example data, and then using the example data of the historical structured query statement as an input prompt of a large language model to assist the large language model to output a result, thereby improving the output accuracy of the large language model.

[0066] In one exemplary embodiment, after obtaining the example data corresponding to the historical structured query statement, the terminal can store the example data corresponding to the historical structured query statement for application of the language model, as shown in Figure 2 The method further includes:

[0067] In step 202, the example data of each historical structured query statement is stored in a database instance to construct an example data set.

[0068] In implementation, the terminal stores the example data of each historical structured query statement in a database instance to construct an example data set. In the application process of the language model, if the model faces an "illusion" problem when understanding and generating natural language, the problem can be effectively alleviated based on Few-shot Prompting, that is, a small amount of example data related to the user query statement is used as an input prompt to assist the language model to output a result.

[0069] In this embodiment, the example data of the historical structured query statement is stored in the database instance, and when a user queries, the relevant example data is quickly found in the example data set of the database instance according to the query request input by the user, thereby avoiding the complex operation of manually generating example data.

[0070] In one exemplary embodiment, the example data corresponding to each historical SQL statement is stored in a database instance, and the terminal integrates a language model, which can be an artificial intelligence algorithm-based question and answer model. In the application process of the language model, as shown in Figure 3 The method further includes:

[0071] Step 302, receiving a query request of a user.

[0072] The query request contains query information in text format.

[0073] In implementation, the user can initiate a query request containing query information in text format in the language model integrated in the terminal. For example, the query request can be a target question in the form of query, such as "get the names of all employees over 50 years old". The terminal receives the query request of the user and runs the language model.

[0074] Step 304, matching target example data associated with the query information in the pre-constructed example data set based on the query information.

[0075] In implementation, the terminal parses and processes the natural language expression query request to obtain the query information in the query request. Then, the terminal matches target example data associated with the query information in the pre-constructed example data set based on the query information. Specifically, the database instance of the terminal stores example data generated by each historical SQL statement. If the current query information corresponds to query content similar to the historical SQL statement, the example data of the historical SQL statement with high similarity can be directly used as target example data and input to the language model for auxiliary output.

[0076] Optionally, if the target example data matching the query information is not found in the pre-constructed example data set, the target example data matching the query information is generated according to the processing process of steps 104 to 106 described above. The generation process of the target example data has been described above and will not be repeated here.

[0077] Step 306, inputting the target example data into the language model to generate a target structured query statement corresponding to the query information.

[0078] In implementation, the target example data contains a natural language description and a SQL statement, i.e., (natural language description, historical SQL). In the above step 302, the query request is "get the names of all employees over 50 years old". If the target example data related to the query information of the query request is "get the names of employees over 30 years old from employees", "SELECT name FROM employees WHERE age > 30", then after obtaining the target example data, the target example data is used as a language model prompt template. The terminal inputs the target example data into the language model, learns the language description method and SQL statement rule of the target example data through the language model, and generates a correct target structured query statement corresponding to the query information input by the user, i.e., generates a correct SQL statement corresponding to the query information: SELECT name FROM employees WHERE age > 50.

[0079] In step 308, based on the language model and the target structured query statement, the query result corresponding to the query request is output.

[0080] In implementation, after obtaining the target SQL statement corresponding to the query information, the terminal analyzes and calculates the target structured query statement (target SQL statement) based on the language model and queries related data, and outputs the query result corresponding to the query request. That is, for the target problem query request input by the user "get the names of all employees over 50 years old", the list of names of all employees over 50 years old in the specific related data set is given as the query result.

[0081] In this embodiment, the historical SQL is converted into example data, thereby forming an example data set. When the user queries, the corresponding example data is found according to the query information input by the user, the example data is added to the prompt as few-shot, and the model generates a correct SQL statement, thereby improving the performance and accuracy of the language model.

[0082] In one exemplary embodiment, as shown in Figure 4 The specific processing process of generating the first hash value based on the historical structured query statement in step 104 includes:

[0083] In step 401, the historical structured query statement is parameterized to obtain a first structured query statement.

[0084] In implementation, the terminal parameterizes the historical structured query statement to obtain the first structured query statement. Specifically, the parameter values in the historical structured query statement are replaced with general markers to obtain the first structured query statement. For example, the historical SQL statement is: SELECT name FROM employees WHERE age > 30. For the historical structured query statement, the "name", "employees" and "age > 30" are replaced with corresponding general markers: ${table_a}, ${field_1}, ${value_1}, etc. The first structured query statement is obtained: SELECT ${field_2} FROM ${table_1} WHERE ${field_1} = ${value_1}. The purpose of parameterizing the historical SQL statement into the first structured query statement is to match the structured query statement template in the database instance when the historical SQL does not generate corresponding sample data.

[0085] In step 402, the first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized to generate a first hash value corresponding to the first structured query statement.

[0086] In implementation, the terminal parses the first structured query statement into an abstract syntax tree ASA (abstract syntax tree), which represents the syntax rules of the programming language in a tree form, and uses nodes to represent language constructs, each node representing a structure in the source code (such as expression, statement, control flow, etc.). Then, the terminal serializes the abstract syntax tree, and generates a first hash value corresponding to the first structured query statement based on the serialized sequence data.

[0087] In this embodiment, by parameterizing and serializing the historical SQL statement, a first hash value of the historical SQL is generated to query and match in the sample data set of the database instance through the first hash value, thereby improving the efficiency of query matching.

[0088] In an exemplary embodiment, as shown in Figure 5 There is a mapping relationship between the structured query statement template and the natural language description template, and the specific processing process of querying the target structured query statement template and the target natural language description template matching the historical structured query statement according to the first hash value in step 104 includes:

[0089] In step 501, among the hash values corresponding to the structured query statement templates, a target hash value consistent with the first hash value is queried.

[0090] In implementation, the hash value of the structured query statement template (i.e. SQL template) and the hash value of the natural language description template are stored in the database instance. In each hash value corresponding to the structured query statement template, the terminal queries the target hash value consistent with the first hash value.

[0091] In step 502, the structured query statement template corresponding to the target hash value is determined as the target structured query statement template matched with the historical structured query statement.

[0092] In implementation, the terminal determines the structured query statement template corresponding to the target hash value as the target structured query statement template matched with the historical SQL statement. Specifically, in the case that the historical SQL statement has no corresponding example data, the corresponding example data is generated based on the structured query statement template corresponding to the historical SQL statement, therefore, the terminal first performs template query based on the first hash value, and queries the target hash value consistent with the first hash value, and then the structured query statement template corresponding to the target hash value is the target structured query statement template matched with the historical SQL statement.

[0093] In step 503, the target natural language description template corresponding to the target structured query statement template is determined based on the mapping relationship between the structured query statement template and the natural language description template.

[0094] In implementation, after the structured query statement template is determined, since the mapping relationship between the structured query statement template and the natural language description template is predefined, the terminal can directly determine the target natural language description template corresponding to the target structured query statement template based on the mapping relationship between the structured query statement template and the natural language description template. In this way, the target structured query statement and the target natural language description template required for generating the example data are determined.

[0095] In this embodiment, the matching is performed through the hash value, the searching is completed in a short time, and the query efficiency is greatly improved. Meanwhile, based on the mapping relationship between the structured query statement template and the natural language description template, the target natural language description template corresponding to the target structured query statement template for generating the example data is quickly determined, and the example data generation efficiency is improved.

[0096] In an exemplary embodiment, in the generation of the example data of the historical SQL, before the example data of the historical SQL is constructed, it can be queried in the database instance whether the example data of the historical SQL already exists, if it exists, it is not necessary to generate, and the next historical SQL statement whose example data is to be generated is directly processed, therefore, before the example data of the historical SQL is generated, the example data of the historical SQL can be verified first, the verification process is still completed by means of the hash value of the historical SQL statement, so as to improve the data query efficiency, as shown in Figure 6 After step 104, the method further comprises:

[0097] Step 601, according to the first hash value, it is queried in each hash value corresponding to the created example data set whether there is a target hash value same as the first hash value.

[0098] In implementation, the hash value of each created example data is stored in the database instance, the terminal queries in each hash value corresponding to the created example data set of the database instance whether there is a target hash value same as the first hash value according to the first hash value of the historical SQL, if the target hash value does not exist, it indicates that the current historical SQL statement does not have corresponding example data, and the generation of the example data of the historical SQL statement is needed, that is, the step of querying the target structured query statement template and the target natural language description template matched with the historical structured query statement according to the first hash value in step 104 in the above embodiment is executed until step 106, and the example data of the historical SQL statement is obtained.

[0099] Step 602, if the target hash value exists, the example data corresponding to the target hash value is determined as the example data of the historical structured query statement.

[0100] In implementation, if the target hash value exists in the example data set, it indicates that the historical SQL statement has corresponding example data in the example data set, therefore, it is not necessary to repeatedly process the historical SQL statement, and the example data corresponding to the target hash value can be directly determined as the example data of the historical structured query statement, that is, the association between the historical SQL and the example data is determined.

[0101] In the embodiment, the data query by means of the first hash value has high efficiency, and if the example data of the historical SQL exists in the database instance, the query is directly based on the first hash value of the historical SQL statement, and it is not necessary to repeatedly generate the example data, so as to save the data processing resource.

[0102] In a specific embodiment, a specific example flow of the generation of the example data based on the data processing method and the application process of the example data in the user query process is given, and the specific example flow is as follows:

[0103] Step one, the SQL template and natural language description template are predefined and stored in the database instance, wherein the SQL template and the natural language description template have a mapping relationship, for example:

[0104] SQL template: SELECT ${field_2} FROM ${table_1} WHERE ${field_1} > ${value_1};

[0105] Natural language description template: Get ${field_2} from ${table_1} where ${field_1} is greater than ${value_1}.

[0106] Step two, get the history SQL statement executed successfully, analyze the history SQL statement:

[0107] (1) Take the history SQL statement: SELECT name FROM employees WHERE age > 30 as an example, parameterize the history SQL statement and replace it with: SELECT $field_1 FROM $table WHERE $field_2 > $value_1;

[0108] (2) Generate AST and serialize the parameterized history SQL statement to get the first hash value.

[0109] (3) Based on the first hash value, match the template in the database instance, confirm the matched SQL template as: "SELECT $field FROM $table WHERE $field > $value;", and the natural language description template as: "Get ${field_2} from ${table_1} where ${field_1} is greater than ${value_1}." Replace the specific value of the natural language description template to form the example data: ("Get name from employees where age is greater than 30", "SELECT name FROM employees WHERE age > 30").

[0110] (4) Add the newly generated example data of the history SQL statement to the example data set of the database instance, and update the example data set.

[0111] Step four, user query processing:

[0112] (1) User query input: "Get all employee names whose age is more than 50."

[0113] (2) The system matches to similar example data in the example data set: (natural language, historical SQL), that is, ("get name of age greater than 30 from employees", "SELECT name FROM employees WHERE age > 30").

[0114] (3) Add this example data to the prompt of the large language model as few-shot, and assist the model to generate the correct SQL statement: SELECT name FROM employees WHERE age > 50.

[0115] It should be understood that although each step in the flowchart involved in the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiments of the present application also provide a data processing apparatus for implementing the above-mentioned data processing method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so the specific limitations in one or more data processing apparatus embodiments provided below can refer to the limitations of the data processing method described above, and will not be repeated here.

[0117] In one exemplary embodiment, as shown in Figure 7 a data processing apparatus 700 is provided, comprising: an acquisition module 701, a matching module 702 and a first generation module 703, wherein:

[0118] The acquisition module 701 is configured to acquire a historical structured query statement.

[0119] The matching module 702 is configured to generate a first hash value based on the historical structured query statement, and query a target structured query statement template and a target natural language description template matched with the historical structured query statement according to the first hash value.

[0120] The first generation module 703 is configured to perform parameter replacement on the target structured query statement template and the target natural language description template based on the historical structured query statement, to generate example data of the historical structured query statement; and the example data is used to assist the language model to output the query result.

[0121] In one of the embodiments, the data processing apparatus 700 further includes:

[0122] The construction module is configured to store the example data of each historical structured query statement into a database instance, to construct an example data set.

[0123] In one of the embodiments, the data processing apparatus 700 further includes:

[0124] The receiving module is configured to receive a query request of a user, and the query request includes query information in a text format;

[0125] The query matching module is configured to match target example data associated with the query information from the pre-constructed example data set based on the query information.

[0126] The second generation module is configured to input the target example data into the language model, to generate a target structured query statement corresponding to the query information.

[0127] The output module is configured to output a query result corresponding to the query request based on the language model and the target structured query statement.

[0128] In one of the embodiments, the matching module 702 is specifically configured to perform parameterization processing on the historical structured query statement, to obtain a first structured query statement.

[0129] The first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized, to generate a first hash value corresponding to the first structured query statement.

[0130] In one of the embodiments, the structured query statement template and the natural language description template have a mapping relationship, and the matching module 702 is specifically configured to query a target hash value consistent with the first hash value from the hash values corresponding to the structured query statement templates.

[0131] The structured query statement template corresponding to the target hash value is determined as the target structured query statement template matched with the historical structured query statement.

[0132] Based on the mapping relationship between the structured query statement template and the natural language description template, the target natural language description template corresponding to the target structured query statement template is determined.

[0133] In one of the embodiments, the apparatus 700 further includes:

[0134] querying, according to the first hash value, whether there is a target hash value identical to the first hash value in each hash value corresponding to the created sample data set;

[0135] determining, if the target hash value exists, that the sample data corresponding to the target hash value is the sample data of the historical structured query statement.

[0136] Each module in the above data processing apparatus can be realized by software, hardware and combinations thereof in whole or in part. Each module can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0137] In an exemplary embodiment, a computer device, which can be a terminal, has an internal structure diagram as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, Near Field Communication (NFC) or other technologies. The computer program is executed by the processor to implement a data processing method. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0138] Those skilled in the art can understand that Figure 8 the structure shown in the above description is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one example embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0140] obtaining historical structured query statements;

[0141] generating a first hash value based on the historical structured query statements, and querying a target structured query statement template and a target natural language description template matched with the historical structured query statements according to the first hash value;

[0142] performing parameter substitution on the target structured query statement template and the target natural language description template based on the historical structured query statements, to generate example data of the historical structured query statements; the example data is used to assist a language model to output query results.

[0143] In one embodiment, the processor further implements the following steps when executing the computer program:

[0144] storing the example data of each historical structured query statement into a database instance, to build an example data set.

[0145] In one embodiment, the processor further implements the following steps when executing the computer program:

[0146] receiving a query request of a user, the query request containing query information in text format;

[0147] matching target example data associated with the query information in the pre-built example data set based on the query information;

[0148] inputting the target example data into a language model to generate a target structured query statement corresponding to the query information;

[0149] outputting a query result corresponding to the query request based on the language model and the target structured query statement.

[0150] In one embodiment, the processor further implements the following steps when executing the computer program:

[0151] performing parameterization processing on the historical structured query statements to obtain a first structured query statement;

[0152] parsing the first structured query statement into an abstract syntax tree, and serializing the abstract syntax tree to generate a first hash value corresponding to the first structured query statement.

[0153] In one embodiment, the processor further implements the following steps when executing the computer program:

[0154] In the hash values corresponding to the structured query statement templates, the target hash value consistent with the first hash value is queried;

[0155] The structured query statement template corresponding to the target hash value is determined as the target structured query statement template matched with the historical structured query statement.

[0156] Based on the mapping relationship between the structured query statement template and the natural language description template, the target natural language description template corresponding to the target structured query statement template is determined.

[0157] In one embodiment, the processor further implements the following steps when executing the computer program:

[0158] According to the first hash value, whether there is a target hash value same as the first hash value in the hash values corresponding to the created sample data set is queried;

[0159] If the target hash value exists, the sample data corresponding to the target hash value is determined as the sample data of the historical structured query statement.

[0160] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the steps in the above method embodiments.

[0161] In one embodiment, a computer program product is provided, and the computer program product includes a computer program. The computer program is executed by the processor to implement the steps in the above method embodiments.

[0162] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., but is not limited thereto. The processor involved in the embodiments provided in the present application can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but is not limited thereto.

[0163] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, it should be considered as the scope of the present application.

[0164] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: Retrieve historical structured query statements; A first hash value is generated based on the historical structured query statement, and a target hash value that matches the first hash value is queried from each hash value corresponding to the structured query statement template. The structured query statement template corresponding to the target hash value is determined as the target structured query statement template that matches the historical structured query statement; Based on the mapping relationship between structured query templates and natural language description templates, the target natural language description template corresponding to the target structured query template is determined; Based on the historical structured query statements, the parameters of the target structured query statement template and the target natural language description template are replaced to generate example data of the historical structured query statements; the example data is used to assist the language model in outputting query results.

2. The method according to claim 1, characterized in that, The method further includes: Example data of each of the aforementioned historical structured query statements are stored in a database instance to construct an example dataset.

3. The method according to claim 2, characterized in that, The method further includes: Receive a user's query request, which contains query information in text format; Based on the query information, target example data associated with the query information is matched in the pre-constructed example dataset; The target example data is input into the language model to generate the target structured query statement corresponding to the query information; Based on the language model and the target structured query statement, the query results corresponding to the query request are output.

4. The method according to claim 1, characterized in that, The generation of the first hash value based on the historical structured query statement includes: The historical structured query statement is parameterized to obtain the first structured query statement; The first structured query statement is parsed into an abstract syntax tree, and the abstract syntax tree is serialized to generate a first hash value corresponding to the first structured query statement.

5. The method according to claim 2, characterized in that, The method further includes: Based on the first hash value, query among the hash values ​​corresponding to the created example dataset whether there is a target hash value that is the same as the first hash value; If the target hash value exists, the example data corresponding to the target hash value will be determined as the example data of the historical structured query statement.

6. A data processing apparatus, characterized in that, The device includes: The retrieval module is used to retrieve historical structured query statements; The matching module is configured to generate a first hash value based on the historical structured query statements, and query a target hash value that matches the first hash value among the hash values ​​corresponding to the structured query statement templates; determine the structured query statement template corresponding to the target hash value as the target structured query statement template that matches the historical structured query statements; and determine the target natural language description template corresponding to the target structured query statement template based on the mapping relationship between the structured query statement template and the natural language description template. The first generation module is used to replace parameters of the target structured query template and the target natural language description template based on the historical structured query statements, and generate example data of the historical structured query statements; the example data is used to assist the language model in outputting query results.

7. The apparatus according to claim 6, characterized in that, The device further includes: The building module is used to store example data of each of the historical structured query statements into a database instance to build the example dataset.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images