Data processing instruction generation, large model training method, device and electronic equipment

Abstract

The present disclosure provides a method, device and electronic equipment for data processing instruction generation and large model training, and relates to the field of computer technology, in particular to the fields of artificial intelligence, databases, natural language processing and large language models. The specific implementation scheme is: obtaining the natural language question information of the current user; obtaining the user rules related to the current user, and determining the user rule description information of the user rules; generating a first prompt information based on the natural language question information, the user rule description information and the preset prompt information template; inputting the first prompt information into the first large language model to obtain the data processing instructions output by the first large language model, and the data processing instructions are used to process the data in the target database. Based on this scheme, the large language model can take the user rules related to the current user into consideration when generating data processing instructions, which helps to improve the accuracy and effectiveness of the generated data processing instructions.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to technical fields such as artificial intelligence, databases, natural language processing, and large language models. Specifically, the present disclosure relates to a method, device, and electronic device for generating data processing instructions and training large models. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing, large language models (LLMs) have achieved remarkable achievements in various tasks.

[0003] Currently, large language models are widely used in code generation tasks. For example, when large language models are applied to generate data processing instructions for a database, users provide natural language information, and the large language model generates data processing instructions for the database based on this natural language information. In this scenario, improving the accuracy and effectiveness of the data processing instructions generated by large language models has become a major technical issue. Summary of the Invention

[0004] In order to solve at least one of the above-mentioned defects, the present disclosure provides a data processing instruction generation, large model training method, device and electronic device.

[0005] According to a first aspect of the present disclosure, a method for generating a data processing instruction is provided, the method comprising:

[0006] Get the current user's natural language question information;

[0007] Obtain user rules related to the current user and determine user rule description information of the user rules;

[0008] Generate first prompt information based on natural language question information, user rule description information and a preset prompt information template;

[0009] The first prompt information is input into the first large language model to obtain a data processing instruction output by the first large language model, where the data processing instruction is used to process data in a target database.

[0010] According to a second aspect of the present disclosure, a large model training method is provided, the method comprising:

[0011] Obtain sample natural language question information from sample users;

[0012] Obtaining sample user rules related to the sample user and determining sample user rule description information of the sample user rules;

[0013] Generate sample first prompt information based on sample natural language question information, sample user rule description information and a preset prompt information template;

[0014] Constructing a training set, which includes sample first prompt information and sample data processing instructions;

[0015] Fine-tune the initial first language model based on the sample data processing instruction to obtain the first language model.

[0016] According to a third aspect of the present disclosure, a data processing instruction generating device is provided, the device comprising:

[0017] Question information acquisition module, used to obtain the natural language question information of the current user;

[0018] A rule description information determination module is used to obtain user rules related to the current user and determine user rule description information of the user rules;

[0019] A prompt information generating module, configured to generate first prompt information based on natural language question information, user rule description information, and a preset prompt information template;

[0020] The data processing instruction generation module is used to input the first prompt information into the first large language model to obtain the data processing instructions output by the first large language model, and the data processing instructions are used to process the data in the target database.

[0021] According to a fourth aspect of the present disclosure, a large model training device is provided, the device comprising:

[0022] A sample question information acquisition module is used to obtain sample natural language question information of sample users;

[0023] A sample rule description information determination module is used to obtain sample user rules related to sample users and determine sample user rule description information of the sample user rules;

[0024] A sample prompt information generation module, configured to generate a sample first prompt information based on the sample natural language question information, the sample user rule description information, and a preset prompt information template;

[0025] A training set construction module is used to construct a training set, which includes sample first prompt information and sample data processing instructions;

[0026] The model training module is used to fine-tune the initial first language model based on the sample data processing instruction to obtain the first language model.

[0027] According to a fifth aspect of the present disclosure, an electronic device is provided, including:

[0028] at least one processor; and

[0029] A memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the above-mentioned at least one processor, and the instructions are executed by the above-mentioned at least one processor so that the above-mentioned at least one processor can execute the above-mentioned data processing instruction generation or large model training method.

[0031] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned data processing instruction generation or large model training method.

[0032] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the above-mentioned data processing instruction generation or large model training method when executed by a processor.

[0033] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0035] Figure 1 This is a flow chart of a method for generating a data processing instruction provided by an embodiment of the present disclosure;

[0036] Figure 2 This is a flow chart of a large model training method provided by an embodiment of the present disclosure;

[0037] Figure 3 1 is a structural diagram of a data processing instruction generating device provided by an embodiment of the present disclosure;

[0038] Figure 4 Schematic diagram of a large model training device provided by an embodiment of the present disclosure;

[0039] Figure 5 A block diagram of an electronic device for implementing the data processing instruction generation or large model training method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0041] Large language models (LLMs) are currently widely used in code generation tasks. For example, when LLMs are applied to generate data processing instructions for a database, users provide natural language information, and the LLMs generate data processing instructions for the database based on that information. In this scenario, improving the accuracy and effectiveness of the data processing instructions generated by LLMs has become a critical technical issue.

[0042] In this scenario, users may have some relevant rules. The large language model in the existing technology cannot take user-related rules into consideration when generating data processing instructions, making it impossible for the large language model to perform personalized reasoning for the user based on user-related rules, affecting the accuracy and effectiveness of the generated data processing instructions.

[0043] In related technologies, the prompt template of a large language model may contain some rule description information. However, this rule description information is generally used to describe some predefined inherent rules. These inherent rules are mostly general rules rather than personalized rules related to the current user. This makes it impossible for the large language model to perform personalized inference for the current user based on the personalized rules related to the current user.

[0044] In related art, when a large language model is applied to a database to generate data processing instructions, the prompt information template generally includes the table structure information of all database tables in the database. When a database contains a large number of database tables, incorporating the table structure information of all database tables into the large language model's prompt information can result in excessive prompt information content, even exceeding the maximum number of tokens supported by the large language model, affecting the normal operation of the large language model. Furthermore, in this case, the table structure of most database tables that are irrelevant to the user's intent will be included in the prompt information, interfering with the large language model's understanding and affecting the accuracy of the data processing instructions generated by the large language model.

[0045] The data processing instruction generation, large model training method, device and electronic device provided by the embodiments of the present disclosure are intended to solve at least one of the above technical problems in the prior art.

[0046] Figure 1A flow chart of a method for generating a data processing instruction according to an embodiment of the present disclosure is shown. Figure 1 As shown in , the method may mainly include:

[0047] Step S110: Obtaining the natural language question information of the current user;

[0048] Step S120: Acquire user rules related to the current user and determine user rule description information of the user rules;

[0049] Step S130: Generate first prompt information based on the natural language question information, the user rule description information, and the preset prompt information template;

[0050] Step S140: Input the first prompt information into a first large language model (LLM) to obtain a data processing instruction output by the first large language model, where the data processing instruction is used to process the data in the target database.

[0051] The natural language question information is text information entered by the current user, used to express the current user's operation intention. For example, "Who are the two students with the highest scores? Give me their parents' phone numbers so I can talk to them."

[0052] User rules are rules related to the current user, and user rule description information is used to describe the user rules. User rules can be personalized rules configured for the current user.

[0053] As an example, user rules related to the current user may be queried based on the user information of the current user.

[0054] The prompt information template may include some information required by the large language model when generating data processing instructions. The prompt information template may include slots corresponding to this information.

[0055] The prompt information template also includes slots corresponding to the natural language question information and the user rule description information, which are used to fill the natural language question information and the user rule description information into the prompt information template respectively, thereby constructing the first prompt information.

[0056] The first large language model may be a large language model in the field of code generation, used to generate data processing instructions based on the first prompt information.

[0057] In the disclosed embodiment, first prompt information can be generated based on natural language question information, user rule description information, and a preset prompt information template. By constructing the first prompt information based on the user rule description information, the large language model can understand the user rules related to the current user and take the user rules into comprehensive consideration. This allows the large language model to perform personalized reasoning for the current user based on the user rules related to the current user, thereby improving the accuracy and effectiveness of the generated data processing instructions.

[0058] In the disclosed embodiment, the data processing instructions are used to process data in the target database. By executing the data processing instructions, operations such as adding (Create), reading (Read), updating (Update), and deleting (Delete) data in the target database can be performed.

[0059] The method provided by the disclosed embodiment obtains natural language question information from the current user; obtains user rules related to the current user and determines user rule description information of the user rules; generates first prompt information based on the natural language question information, the user rule description information, and a preset prompt information template; inputs the first prompt information into a first large language model, and obtains data processing instructions output by the first large language model, which are used to process data in a target database. Based on this solution, the large language model can take user rules related to the current user into consideration when generating data processing instructions, which helps to improve the accuracy and effectiveness of the generated data processing instructions.

[0060] In an optional manner of the present disclosure, obtaining user rules related to the current user includes:

[0061] Obtain at least one data permission rule for the current user to the data in the target database;

[0062] Determine the user rule description information of the user rule, including:

[0063] Generate data permission rule description information corresponding to each data permission rule;

[0064] Generate user rule description information based on data permission rule description information.

[0065] In the embodiment of the present disclosure, user rule description information can be used to describe the current user's data permission rules for data in the target database, so that the large language model can understand the current user's data permission rules based on the user rule description information, avoid generating data processing instructions that do not meet the current user's data permission rules, and improve the accuracy and effectiveness of data processing instructions.

[0066] For example, when the current user does not have the permission to modify a database table, the large language model can understand the current user's data permission rules based on the user rule description information and will not generate data processing instructions for modifying the database table, thereby ensuring the security of the database table.

[0067] In the disclosed embodiment, the data permission rules are used to indicate the operation permissions that the current user has on the data in the target database, such as the operation permissions for operations such as adding (Create), reading (Read), updating (Update) and deleting (Delete).

[0068] When obtaining the data permission rules of the current user for data in the target database, the user information of the current user can be obtained, and the data permission rules of the current user can be queried based on the user information of the current user.

[0069] As an example, data permission rules are generally managed based on user groups or user roles. Therefore, the user group or user role to which the current user belongs can be determined based on the user information of the current user, thereby determining the data permission rules of the current user based on the user group or user role to which the current user belongs.

[0070] As an example, data permission rules can include database table permission rules and field permission rules. Table permission rules indicate the current user's permission to operate on a database table within the target database. Field permission rules indicate the current user's permission to operate on a field within a database table within the target database.

[0071] In the embodiment of the present disclosure, after obtaining at least one data permission rule of the current user, data permission rule description information may be generated for each data permission rule.

[0072] For example, suppose the current user is user X, and they have two data permission rules: one for viewing (reading) database table c, and the other for deleting field e in database table d. In this case, the data permission rule description could be "Allow user Y to view database table c; allow user Y to delete field e in database table d."

[0073] In the embodiment of the present disclosure, after data permission rule description information is generated for each data permission rule, user rule description information may be generated based on the data permission rule description information.

[0074] As an example, the data permission rule description information corresponding to the data permission rules may be summarized into a data permission rule description information list, and all the data permission rule description information in the list may be used as the user rule description information.

[0075] It is understandable that the user rules described in the user rule description information in the embodiment of the present disclosure may include, in addition to data permission rules, some other user rules related to the current user. It is only necessary to ensure that the description information corresponding to these user rules can help the large language model better generate data processing instructions. As an example, the user rule is to prohibit the generation of data processing instructions for the current user within a specified time period, the specified time period is from time a to time b, and the current user is user X. The user rule description information can be "Prohibit the generation of data processing instructions for user X from time a to time b".

[0076] In an optional manner of the present disclosure, the data processing instruction is a Structured Query Language (SQL) instruction.

[0077] In the embodiment of the present disclosure, the data processing instruction may be an SQL instruction, that is, the database executes an SQL statement.

[0078] It is understandable that the data processing instructions involved in the embodiments of the present disclosure are not limited to SQL instructions, but may also include other types of instructions.

[0079] In an optional manner of the present disclosure, the data permission rule description information includes multiple items, and user rule description information is generated based on the data permission rule description information, including:

[0080] Merge multiple data permission rule description information to obtain user rule description information.

[0081] In the disclosed embodiment, the current user may have multiple data permission rules, and accordingly, the data permission rule description information will also include multiple items. If the multiple data permission rule description information is directly used as the user rule description information and filled in the prompt information template, it will result in too much content in the prompt information, and even exceed the maximum number of tokens supported by the large language model, affecting the normal operation of the large language model.

[0082] In the disclosed embodiment, multiple data permission rule description information can be merged to obtain user rule description information, that is, the user rule description information can be used to merge and describe multiple data permission rules. By merging the data permission rule description information, the text content contained in the user rule description information can be made more concise, avoiding the situation where the prompt information contains too much content, or even exceeds the maximum number of tokens supported by the large language model, and ensuring the normal operation of the large language model.

[0083] In the embodiment of the present disclosure, a merging rule may be set, and the data permission rule description information may be merged according to the merging rule.

[0084] As an example, the merge rule may be: when a user has a field data permission rule for all fields in a database table, the field permission rule description information for each field in the database table is merged to obtain the user's data permission rule description information for the database table.

[0085] As an example, database table T contains fields T1, T2, and T3. The current user is user X, who has view permission for fields T1, T2, and T3. The field permission rule description information for each field is "User X has view permission for field T1; User X has view permission for field T2; User X has view permission for field T3." Based on the above merge rule, the field permission rule description information is merged to obtain the data permission rule description information for database table T: "User X has view permission for database table T."

[0086] In an optional manner of the present disclosure, multiple data permission rule description information is merged to obtain user rule description information, including:

[0087] Determining second prompt information based on table structure information of each database table in the target database and description information of each data permission rule;

[0088] The second prompt information is input into the second largest language model, so that the second largest language model merges the multiple data authority rule description information to obtain user rule description information.

[0089] In the disclosed embodiment, the data authority rule description information may be merged based on the reasoning capability of the second largest language model.

[0090] Specifically, the second prompt information can be constructed based on the table structure information of each database table in the target database, so that the second largest language model can understand the data structure in the target database, thereby merging multiple data permission rule description information to simplify the expression of user rule description information.

[0091] In the disclosed embodiment, the second prompt information of the second largest language model may be constructed based on the table structure information of each database table in the target database and the description information of each data permission rule.

[0092] For example, the prompt template for the second language model might include the table structure information for each database table in the target database, descriptions of multiple data permission rules for the current user, and a template question. The template question might be something like, "As a database expert, please analyze the user's multiple data permission rule descriptions and combine them into a concise rule description."

[0093] In an optional embodiment of the present disclosure, the prompt information template further includes:

[0094] Table structure information of the target database table;

[0095] The target database table is a database table stored in the target database.

[0096] In the embodiment of the present disclosure, the prompt information template may further include the table structure information of the target database table, so that the prompt information constructed based on the prompt information template contains the table structure information of the target database table, so that the first language model can generate data processing instructions based on the table structure information of the target database table.

[0097] As an example, the target database table may be all or part of the database tables stored in the target database.

[0098] In an optional manner of the present disclosure, the target database table is a database table in the target database whose correlation with the natural language question information meets a preset correlation condition.

[0099] In the embodiment of the present disclosure, the preset correlation condition can be set according to actual needs. When the target database table and the natural language question information meet the preset correlation condition, it means that the target database table is a database table related to the current user's operation intention.

[0100] By extracting the target database table relevant to the current user's operational intent from all database tables in the target database based on the correlation with the natural language question information, and incorporating the target database table's table structure information into the prompt information, this method avoids the excessive content in the prompt information that would result from incorporating the table structure information of all database tables in the target database into the prompt information, preventing the maximum number of tokens supported by the large language model from being exceeded, and ensuring the normal operation of the large language model. Furthermore, because the target database table is the database table relevant to the current user's operational intent, the table structures of most database tables unrelated to the user's operational intent are not included in the prompt information, preventing interference with the large language model's understanding and ensuring the accuracy of the data processing instructions generated by the large language model.

[0101] As an example, the semantic similarity between the table name of a database table and the natural language question information can be determined, and the semantic similarity can be used as the correlation between the database table of the target database and the natural language question information. The preset correlation condition is: the semantic similarity between the table name of the database table and the natural language question information is greater than a preset semantic similarity threshold.

[0102] In an optional embodiment of the present disclosure, the above method further includes:

[0103] The third language model is called to determine the target database table from the database tables of the target database.

[0104] In the disclosed embodiment, the semantic understanding capability of the third language model can be used to analyze the correlation between natural language question information and each database table in the target database, and determine the target database table from the database tables in the target database.

[0105] Specifically, the table structure information of each database table in the target database and the natural language question information can be input into the third language model, so that the third language model filters out the target database table from the database tables of the target database.

[0106] As an example, the prompt information template of the third language model may include a database table structure, user questions, template prompt questions, and an output format. Among them, the database table structure is the table structure information of all database tables in the target database, and the user questions are the natural language questions asked by the current user. The template question can be "You are an SQL expert. Please analyze the user's question and determine whether there is a query intention for the following database table. Please output the table name that the user wants to query in JSON format." The output format is used to indicate the output format of the output result of the third language model. For example, the output format is: "table" "queried table name" # Note, if there are multiple tables, please use ',' to concatenate, and use an empty string to represent them if they do not exist. This output format indicates that if the target database table exists, the table name of the database table can be output, and if there are multiple target databases, the multiple target database tables are separated by commas; if the target database table does not exist, an empty character is output.

[0107] In an optional embodiment of the present disclosure, the prompt information template further includes:

[0108] Pre-set inherent rule description information.

[0109] In the embodiment of the present disclosure, the prompt information template further includes some inherent rule description information, which is used to describe the inherent rules. The inherent rules can be understood as some predefined general rules.

[0110] As an example, the inherent rule description information may be: "Prohibit querying tables that are not in the table structure definition", which means that the large language model will not generate data processing instructions for database tables that do not exist in the target database.

[0111] As an example, the inherent rule description information may be: “prohibit querying table structure”, which means that the large language model will not generate a data processing instruction for querying the table structure in the target database.

[0112] As an example, an inherent rule description could be: "Prohibit explicit SQL statements from being included in user questions." When a user's natural language question contains explicit SQL statements, these SQL statements may skip the large language model's reasoning process and be directly output as data query instructions. In this case, the generated data processing instructions have not been reasoned by the large language model, and the security of the data processing instructions cannot be guaranteed, potentially leading to data security risks. By setting inherent rules to prohibit explicit SQL statements from being included in natural language questions, the large language model will not generate data processing instructions for this type of natural language question information, thereby ensuring the security of the data processing instructions.

[0113] In an optional embodiment of the present disclosure, the prompt information template further includes:

[0114] Output format specifying information, the output format specifying information is related to at least one of the user rule description information and the inherent rule description information.

[0115] In the disclosed embodiment, the prompt information template further includes output format specifying information, which is used to indicate the output format of the output result of the first language model, so as to facilitate the subsequent use of the output result.

[0116] In an embodiment of the present disclosure, the output format specification information is related to at least one of the user rule description information and the inherent rule description information, that is, the output format of the output result is set according to the user rule description information and / or the inherent rule description information, so as to facilitate understanding the output result according to the user rules or the inherent rules.

[0117] As an example, the output format specifies the information as follows: "SQL": "Generated SQL statement", #Note: An empty string is used to indicate when the rules are not satisfied; "reason": "Content of the rule that is not satisfied"", #Note: An empty string is used to indicate when the rules are satisfied. This output format specifies the information indicating that the output result includes a column corresponding to the SQL statement and a column corresponding to the rule. When the user rule description and the inherent rules are satisfied, the SQL statement is output in the column corresponding to the SQL statement, and an empty character is output in the column corresponding to the rule. When the user rule description and the inherent rules are not satisfied, an empty character is output in the column corresponding to the SQL statement, and the specific content of the unsatisfied user rule or the specific content of the unsatisfied inherent rule is output in the column corresponding to the rule.

[0118] In an optional embodiment of the present disclosure, the prompt information template further includes:

[0119] Preset template question information.

[0120] In the disclosed embodiment, the prompt information template may further include preset template question information for effectively guiding the first language model.

[0121] As an example, the template question information may be "You are an SQL expert, please generate a syntactically correct SQL statement based on the given user question."

[0122] As an example, a prompt information template provided by an embodiment of the present disclosure includes a rule list (rules), a table structure definition (tables, a user question (query), and an output format (format). The rule list may include a list consisting of user rule description information and inherent rule description information. The table structure definition is the table structure information of the target database table. The user question is the natural language question information of the current user. The output format is the output format specification information.

[0123] In an optional method of the present disclosure, obtaining the natural language question information of the current user includes:

[0124] Obtain the user's initial natural language question information;

[0125] The initial natural language question information is semantically optimized to obtain natural language question information.

[0126] In the embodiment of the present disclosure, the initial natural language question information is the question information initially input by the user. The initial natural language question information may contain ambiguous meanings, which may affect the understanding of the large language model.

[0127] In the embodiment of the present disclosure, the initial natural language question information can be semantically optimized to obtain natural language question information. The semantics of the natural language question information is optimized, which can avoid ambiguous meanings. The prompt information of the first large language model is constructed using the natural language question information, which helps to understand the large language model.

[0128] In the embodiment of the present disclosure, semantic optimization processing may include, but is not limited to, typo optimization, information redundancy optimization, and semantic integrity optimization.

[0129] In the disclosed embodiment, a large language model may be called to utilize the semantic understanding capability of the large language model to perform semantic optimization processing on the initial natural language question information.

[0130] In an optional embodiment of the present disclosure, after obtaining the data processing instruction output by the first language model, the method further includes:

[0131] The data in the target database is processed based on the data processing instructions to obtain data processing results.

[0132] In the embodiment of the present disclosure, the data processing instructions output by the first language model can be used to process the data in the target database to obtain data processing results.

[0133] For example, the data processing instruction is an SQL instruction for the target database, and the SQL instruction can be executed to implement operations such as adding, reading, updating and deleting on the target database.

[0134] In an optional manner of the present disclosure, processing data in a target database based on a data processing instruction includes:

[0135] Determine whether the data permission rules of the current user support the data processing instructions;

[0136] In response to the data permission rule of the current user supporting the data processing instruction, the data in the target database is processed based on the data processing instruction.

[0137] In the disclosed embodiment, the data processing instructions output by the large language model may be somewhat unstable. The data processing instructions output by the large language model may be verified to ensure the security of the data processing instructions. The data processing instructions may be executed after the verification is passed.

[0138] Specifically, the data processing instructions can be verified based on the current user's data permission rules to determine whether the current user's data permission rules support the data processing instructions. After determining that the current user's data permission rules support the data processing instructions, the data in the target database can be processed based on the data processing instructions.

[0139] As an example, keyword screening rules can be set according to the data permission rules of the current user. For example, if the current user does not have the permission to modify the database table or the fields in the database table, the keyword screening rules can be set so that the data processing instructions do not contain instruction keywords related to the modification operation. Instruction keywords related to the modification operation include "CREATE", "INSERT", "ALTER", "UPDATE", "DROP", "DELETE", "MERGE", etc.

[0140] Figure 2 A flow chart of a large model training method provided by an embodiment of the present disclosure is shown as follows: Figure 2 As shown in , the method may mainly include:

[0141] Step S210: Obtain sample natural language question information of sample users;

[0142] Step S220: obtaining sample user rules related to the sample user, and determining sample user rule description information of the sample user rules;

[0143] Step S230: Generate sample first prompt information based on the sample natural language question information, the sample user rule description information and the preset prompt information template;

[0144] Step S240: constructing a training set, the training set including the sample first prompt information and the sample data processing instructions;

[0145] Step S250: Fine-tune the initial first language model based on the sample data processing instruction to obtain the first language model.

[0146] The sample natural language question information is text information input by the sample user, and is used to express the sample user's operation intention.

[0147] The sample user rules are rules related to the sample users, and the sample user rule description information is used to describe the sample user rules. The sample user rules can be personalized rules configured for the sample users.

[0148] The prompt information template may include some information required by the large language model when generating data processing instructions. The prompt information template may include slots corresponding to this information.

[0149] The prompt information template also includes corresponding slots for sample natural language question information and sample user rule description information, which are used to fill the sample natural language question information and sample user rule description information into the prompt information template respectively, thereby constructing the sample first prompt information.

[0150] In the embodiment of the present disclosure, sample first prompt information can be generated based on sample natural language question information, sample user rule description information, and a preset prompt information template.

[0151] In the embodiment of the present disclosure, a training set can be constructed based on the sample first prompt information and the sample data processing instructions, wherein the sample data processing instructions can be used as labels, which are data processing instructions obtained by analyzing and reasoning based on the sample first prompt information.

[0152] The initial first large language model may be a pre-trained large prediction model of a large language model suitable for the field of code generation.

[0153] In the embodiment of the present disclosure, the initial first large language model may be fine-tuned and trained based on the sample data processing instruction to obtain the first large language model.

[0154] The method provided by the embodiment of the present disclosure obtains sample natural language question information of a sample user; obtains sample user rules related to the sample user and determines sample user rule description information of the sample user rules; generates sample first prompt information based on the sample natural language question information, the sample user rule description information, and a preset prompt information template; constructs a training set, the training set including the sample first prompt information and sample data processing instructions; and fine-tunes the initial first large language model based on the sample data processing instructions to obtain the first large language model. The large prediction model trained based on this solution can take into account the user rules related to the current user when generating data processing instructions, which helps to improve the accuracy and effectiveness of the generated data processing instructions.

[0155] Based on Figure 1 The same principle as shown in the method, Figure 3 A structural diagram of a data processing instruction generating device provided by an embodiment of the present disclosure is shown as follows: Figure 3As shown, the data processing instruction generating device 30 may include:

[0156] The question information acquisition module 310 acquires the natural language question information of the current user;

[0157] The rule description information determination module 320 obtains user rules related to the current user and determines user rule description information of the user rules;

[0158] The prompt information generation module 330 generates first prompt information based on the natural language question information, the user rule description information and the preset prompt information template;

[0159] The data processing instruction generation module 340 inputs the first prompt information into the first large language model to obtain the data processing instruction output by the first large language model, where the data processing instruction is used to process the data in the target database.

[0160] The device provided by the disclosed embodiment obtains natural language question information from the current user; obtains user rules related to the current user and determines user rule description information of the user rules; generates first prompt information based on the natural language question information, the user rule description information, and a preset prompt information template; inputs the first prompt information into a first large language model, and obtains data processing instructions output by the first large language model, which are used to process data in a target database. Based on this solution, the large language model can take user rules related to the current user into consideration when generating data processing instructions, which helps to improve the accuracy and effectiveness of the generated data processing instructions.

[0161] Optionally, when acquiring user rules related to the current user, the rule description information determination module is specifically configured to:

[0162] Obtain at least one data permission rule for the current user to the data in the target database;

[0163] When determining the user rule description information of a user rule, the rule description information determination module is specifically used to:

[0164] Generate data permission rule description information corresponding to each data permission rule;

[0165] Generate user rule description information based on data permission rule description information.

[0166] Optionally, the data permission rule description information includes multiple items. When the rule description information determination module generates the user rule description information based on the data permission rule description information, it is specifically used to:

[0167] Merge multiple data permission rule description information to obtain user rule description information.

[0168] Optionally, when the rule description information determination module merges multiple pieces of data permission rule description information to obtain user rule description information, it is specifically used to:

[0169] Determining second prompt information based on table structure information of each database table in the target database and description information of each data permission rule;

[0170] The second prompt information is input into the second largest language model, so that the second largest language model merges the multiple data authority rule description information to obtain user rule description information.

[0171] Optionally, the prompt information template also includes:

[0172] Table structure information of the target database table;

[0173] The target database table is a database table stored in the target database.

[0174] Optionally, the target database table is a database table in the target database whose correlation with the natural language question information satisfies a preset correlation condition.

[0175] Optionally, the above device further includes:

[0176] The target database table determination module is used to call the third language model to determine the target database table from the database tables of the target database.

[0177] Optionally, the prompt information template also includes:

[0178] Pre-set inherent rule description information.

[0179] Optionally, the prompt information template also includes:

[0180] Output format specifying information, where the output format specifying information is related to at least one of the user rule description information and the inherent rule description information.

[0181] Optionally, the prompt information template also includes:

[0182] Preset template question information.

[0183] Optionally, the question information acquisition module is specifically used to:

[0184] Obtain the user's initial natural language question information;

[0185] The initial natural language question information is semantically optimized to obtain natural language question information.

[0186] Optionally, the above device further includes:

[0187] The data processing module is used to process the data in the target database based on the data processing instructions after obtaining the data processing instructions output by the first language model to obtain data processing results.

[0188] Optionally, the data processing module is specifically used to:

[0189] Determine whether the data permission rules of the current user support the data processing instructions;

[0190] In response to the data permission rule of the current user supporting the data processing instruction, the data in the target database is processed based on the data processing instruction.

[0191] Optionally, the data processing instruction is an SQL instruction.

[0192] It is understandable that the above modules of the data processing instruction generating device in the embodiment of the present disclosure have the function of realizing Figure 1 The functions of the corresponding steps of the data processing instruction generation method in the embodiment shown in . This function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and the above modules can be implemented separately or integrated with multiple modules. For the functional description of each module of the above data processing instruction generation device, please refer to Figure 1 The corresponding description of the data processing instruction generation method in the embodiment shown in will not be repeated here.

[0193] Based on Figure 4 The same principle as shown in the method, Figure 4 A schematic diagram of the structure of a large model training device provided by an embodiment of the present disclosure is shown. Figure 4 As shown, the large model training device 40 may include:

[0194] The sample question information acquisition module 410 is used to obtain sample natural language question information of sample users;

[0195] The sample rule description information determination module 420 is configured to obtain sample user rules related to the sample user and determine sample user rule description information of the sample user rules;

[0196] The sample prompt information generating module 430 is used to generate sample first prompt information based on the sample natural language question information, the sample user rule description information and the preset prompt information template;

[0197] A training set construction module 440 is used to construct a training set, wherein the training set includes sample first prompt information and sample data processing instructions;

[0198] The model training module 450 is configured to perform fine-tuning training on the initial first language model based on the sample data processing instructions to obtain the first language model.

[0199] The device provided by the embodiment of the present disclosure obtains sample natural language question information of a sample user; obtains sample user rules related to the sample user and determines sample user rule description information of the sample user rules; generates sample first prompt information based on the sample natural language question information, the sample user rule description information and a preset prompt information template; constructs a training set, the training set includes the sample first prompt information and sample data processing instructions; fine-tunes the initial first large language model based on the sample data processing instructions to obtain the first large language model. The large prediction model trained based on this solution can take into account the user rules related to the current user when generating data processing instructions, which helps to improve the accuracy and effectiveness of the generated data processing instructions.

[0200] It is understandable that the above modules of the large model training device in the embodiment of the present disclosure have the function of realizing Figure 2 The functions of the corresponding steps of the large model training method in the embodiment shown in . This function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and the above modules can be implemented separately or integrated with multiple modules. For the functional description of each module of the above large model training device, please refer to Figure 2 The corresponding description of the large model training method in the embodiment shown in the will not be repeated here. …

[0201] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0202] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0203] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing instruction generation or large model training method provided in the embodiments of the present disclosure.

[0204] Compared to existing technologies, this electronic device obtains natural language question information from the current user; obtains user rules related to the current user and determines user rule description information for the user rules; generates first prompt information based on the natural language question information, user rule description information, and a preset prompt information template; and inputs the first prompt information into a first large language model to obtain data processing instructions output by the first large language model. The data processing instructions are used to process data in a target database. Based on this solution, the large language model can take user rules related to the current user into consideration when generating data processing instructions, which helps improve the accuracy and effectiveness of the generated data processing instructions.

[0205] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute a data processing instruction generation or large model training method as provided in an embodiment of the present disclosure.

[0206] Compared to existing technologies, this readable storage medium obtains natural language question information from the current user; obtains user rules related to the current user and determines user rule description information for the user rules; generates first prompt information based on the natural language question information, user rule description information, and a preset prompt information template; and inputs the first prompt information into a first large language model to obtain data processing instructions output by the first large language model. The data processing instructions are used to process data in a target database. Based on this solution, the large language model can take user rules related to the current user into consideration when generating data processing instructions, which helps improve the accuracy and effectiveness of the generated data processing instructions.

[0207] The computer program product includes a computer program, which, when executed by a processor, implements the data processing instruction generation or large model training method provided in the embodiments of the present disclosure.

[0208] Compared to existing technologies, this computer program product obtains natural language question information from the current user; obtains user rules related to the current user and determines user rule description information for the user rules; generates first prompt information based on the natural language question information, the user rule description information, and a preset prompt information template; and inputs the first prompt information into a first large language model to obtain data processing instructions output by the first large language model. The data processing instructions are used to process data in a target database. Based on this solution, the large language model can take the user rules related to the current user into consideration when generating data processing instructions, which helps improve the accuracy and effectiveness of the generated data processing instructions.

[0209] Figure 5A schematic block diagram of an example electronic device 50 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0210] like Figure 5 As shown, the electronic device 50 includes a computing unit 510, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 520 or a computer program loaded from a storage unit 580 into a random access memory (RAM) 530. Various programs and data required for the operation of the device 50 can also be stored in the RAM 530. The computing unit 510, the ROM 520, and the RAM 530 are connected to each other via a bus 540. An input / output (I / O) interface 550 is also connected to the bus 540.

[0211] Multiple components in device 50 are connected to I / O interface 550, including: an input unit 560, such as a keyboard, mouse, etc.; an output unit 570, such as various types of displays, speakers, etc.; a storage unit 580, such as a magnetic disk, optical disk, etc.; and a communication unit 590, such as a network card, modem, wireless communication transceiver, etc. The communication unit 590 allows device 50 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0212] The computing unit 510 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 510 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 510 performs the data processing instruction generation or large model training method provided in the embodiments of the present disclosure. For example, in some embodiments, the execution of the data processing instruction generation or large model training method provided in the embodiments of the present disclosure can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 580. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 50 via the ROM 520 and / or the communication unit 590. When the computer program is loaded into the RAM 530 and executed by the computing unit 510, one or more steps of the data processing instruction generation or large model training method provided in the embodiments of the present disclosure can be performed. Alternatively, in other embodiments, the computing unit 510 may be configured in any other appropriate manner (e.g., by means of firmware) to execute the data processing instruction generation or large model training method provided in the embodiments of the present disclosure.

[0213] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0214] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0215] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0216] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0217] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0218] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0219] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0220] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for generating a data processing instruction, comprising: Get the current user's natural language question information; Acquire user rules related to the current user and determine user rule description information of the user rules; Generate first prompt information based on the natural language question information, the user rule description information and a preset prompt information template; Inputting the first prompt information into a first large language model to obtain a data processing instruction output by the first large language model, wherein the data processing instruction is used to process data in a target database; The acquiring of user rules related to the current user includes: Obtaining at least one data permission rule of the current user for data in the target database; The user rule description information for determining the user rule includes: Generate data permission rule description information corresponding to each data permission rule, wherein the data permission rule description information includes multiple pieces of information; Determining second prompt information based on table structure information of each database table in the target database and description information of each data permission rule; The second prompt information is input into a second language model, so that the second language model merges the multiple data authority rule description information to obtain the user rule description information.

2. The method according to claim 1, wherein The prompt information template also includes: Table structure information of the target database table; The target database table is a database table stored in the target database.

3. The method according to claim 2, wherein: The target database table is a database table in the target database whose correlation with the natural language question information meets a preset correlation condition.

4. The method according to claim 3, further comprising: The third language model is called to determine the target database table from database tables of the target database.

5. The method according to any one of claims 1 to 4, wherein The prompt information template also includes: Pre-set inherent rule description information.

6. The method according to claim 5, wherein: The prompt information template also includes: Output format designation information is associated with at least one of user rule description information and inherent rule description information.

7. The method according to any one of claims 1 to 4, wherein The prompt information template also includes: Preset template question information.

8. The method according to any one of claims 1 to 4, wherein The obtaining of the natural language question information of the current user includes: Obtain the user's initial natural language question information; The initial natural language question information is subjected to semantic optimization processing to obtain the natural language question information.

9. The method according to any one of claims 1 to 4, wherein After obtaining the data processing instruction output by the first language model, the method further includes: The data in the target database is processed based on the data processing instruction to obtain a data processing result.

10. The method according to claim 9, wherein: The processing of the data in the target database based on the data processing instruction includes: Determining whether the data permission rules of the current user support the data processing instruction; In response to the data permission rule of the current user supporting the data processing instruction, the data in the target database is processed based on the data processing instruction.

11. The method according to any one of claims 1 to 10, wherein The data processing instructions are structured query language SQL instructions.

12. A large model training method comprising: Obtain sample natural language question information from sample users; Acquire sample user rules related to the sample user, and determine sample user rule description information of the sample user rules; Generate sample first prompt information based on the sample natural language question information, the sample user rule description information and a preset prompt information template; Constructing a training set, wherein the training set includes the sample first prompt information and the sample data processing instruction; Fine-tune the initial first language model based on the sample data processing instruction to obtain the first language model; The acquiring of sample user rules related to the sample user includes: Obtaining at least one sample data permission rule of the sample user for data in the target database; The sample user rule description information for determining the sample user rule includes: generating sample data permission rule description information corresponding to each sample data permission rule, wherein the sample data permission rule description information includes a plurality of pieces; Determining second prompt information based on table structure information of each database table in the target database and description information of permission rules of each sample data; The second prompt information is input into a second language model, so that the second language model merges the plurality of sample data authority rule description information to obtain the sample user rule description information.

13. A data processing instruction generating device, comprising: Question information acquisition module, used to obtain the natural language question information of the current user; A rule description information determination module, configured to obtain user rules related to the current user and determine user rule description information of the user rules; a prompt information generating module, configured to generate first prompt information based on the natural language question information, the user rule description information, and a preset prompt information template; a data processing instruction generation module, configured to input the first prompt information into a first large language model to obtain a data processing instruction output by the first large language model, wherein the data processing instruction is used to process data in a target database; When acquiring the user rules related to the current user, the rule description information determination module is specifically used to: Obtaining at least one data permission rule of the current user for data in the target database; When determining the user rule description information of the user rule, the rule description information determination module is specifically used to: Generate data permission rule description information corresponding to each data permission rule, wherein the data permission rule description information includes multiple pieces of information; Determining second prompt information based on table structure information of each database table in the target database and description information of each data permission rule; The second prompt information is input into the second largest language model, so that the second largest language model merges the multiple data authority rule description information to obtain the user rule description information.

14. A large model training device comprising: A sample question information acquisition module is used to obtain sample natural language question information of sample users; a sample rule description information determination module, configured to obtain sample user rules related to the sample user and determine sample user rule description information of the sample user rules; A sample prompt information generation module, configured to generate sample first prompt information based on the sample natural language question information, the sample user rule description information, and a preset prompt information template; A training set construction module, configured to construct a training set, wherein the training set includes the sample first prompt information and a sample data processing instruction; A model training module, configured to perform fine-tuning training on the initial first language model based on the sample data processing instruction to obtain the first language model; The sample rule description information determination module is configured to obtain sample user rules related to the sample user, including: The sample rule description information determination module is used to obtain at least one sample data permission rule of the sample user for data in the target database; The sample rule description information determining module is configured to determine the sample user rule description information of the sample user rule, including: The sample rule description information determination module is configured to generate sample data permission rule description information corresponding to each sample data permission rule, wherein the sample data permission rule description information includes multiple pieces of information; determine second prompt information based on the table structure information of each database table in the target database and each sample data permission rule description information; input the second prompt information into a second large language model, so that the second large language model merges the multiple pieces of sample data permission rule description information to obtain the sample user rule description information.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-12.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images