Structured query language code evaluation method and device, medium, equipment and product

Abstract

A structured query language code assessment method and apparatus, a medium, a device and a product, the method comprising: obtaining first query information and a first structured query language SQL code to be assessed, the first SQL code being used for querying an answer corresponding to the first query information; determining a first evaluation result of the first SQL code through at least one agent; wherein the intelligent agent is used for determining the first evaluation result according to preset information or a large language model. According to the technical scheme, different agents can evaluate the SQL code in different modes, the multiple agents work cooperatively, the evaluation accuracy can be improved, and the expandability of the SQL code evaluation method can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method, apparatus, medium, device, and product for evaluating Structured Query Language (SQL) code. Background Art

[0002] Data query is applied to various business scenarios. Data is usually stored in database tables. When performing a data query, a Structured Query Language (SQL) needs to be used for querying. The accuracy of the SQL code used for querying in a database affects the accuracy of the query result. Therefore, it is very important to evaluate the SQL code. Summary of the Invention

[0003] This Summary of the Invention section is provided to introduce concepts in a brief form, which will be described in detail in the following Detailed Implementation section. This Summary of the Invention section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a method for evaluating Structured Query Language (SQL) code, the method including: Obtaining first query information and a first Structured Query Language (SQL) code to be evaluated, where the first SQL code is used to query an answer corresponding to the first query information; Determining a first evaluation result for the first SQL code through at least one agent; where the agent is used to determine the first evaluation result according to preset information or a large language model.

[0005] In a second aspect, the present disclosure provides an apparatus for evaluating Structured Query Language (SQL) code, the apparatus including: An obtaining module, configured to obtain first query information and a first Structured Query Language (SQL) code to be evaluated, where the first SQL code is used to query an answer corresponding to the first query information; A determining module, configured to determine a first evaluation result for the first SQL code through at least one agent; where the agent is used to determine the first evaluation result according to preset information or a large language model.

[0006] In a third aspect, the present disclosure provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method for evaluating Structured Query Language (SQL) code provided in the first aspect of the present disclosure are implemented.

[0007] In a fourth aspect, the present disclosure provides an electronic device, including: A storage device, on which a computer program is stored; A processing device for executing the computer program in the storage device to implement the steps of the structured query language code evaluation method provided in the first aspect of the present disclosure.

[0008] In a fifth aspect, the present disclosure provides a computer program product including a computer program that, when executed by a processor, implements the steps of the structured query language code evaluation method provided in the first aspect of the present disclosure.

[0009] By adopting the above technical solution, the first evaluation result of the first SQL code is determined by at least one agent, and the agent is used to determine the first evaluation result according to preset information or a large language model. In this way, multiple agents can be preset, and the way for the agent to evaluate the SQL code can be set. For example, the agent can evaluate the first SQL code according to preset information, and the agent can evaluate the first SQL code according to the large language model. That is, different agents can evaluate the SQL code in different ways, and the evaluation of the SQL code can be based on a multi-agent architecture. Multiple agents work together to improve the accuracy of the evaluation. Moreover, the number of agents can be set according to the evaluation requirements, and the evaluation method adopted by the agent can also be set according to the evaluation requirements, improving the scalability of the SQL code evaluation method.

[0010] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In combination with the drawings and with reference to the following specific implementation, the above and other features, advantages and aspects of the various embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original elements and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a structured query language code evaluation method shown according to an exemplary embodiment.

[0012] Figure 2 is an architecture diagram of an agent shown according to an exemplary embodiment.

[0013] Figure 3 is a flowchart of a method for determining the first evaluation result of the first SQL code shown according to an exemplary embodiment.

[0014] Figure 4 is a flowchart of a method for determining the first evaluation result of the first SQL code shown according to another exemplary embodiment.

[0015] Figure 5It is a block diagram of a structured query language code evaluation device shown according to an exemplary embodiment.

[0016] Figure 6 It shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. Detailed implementation manners

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0018] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0019] The term "including" and its variations used herein are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions executed by these devices, modules or units or their interdependent relationships.

[0021] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to users and the authorization of users should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0024] For example, when responding to an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operations of the present disclosure technical solution according to the prompt message.

[0025] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, a pop-up window manner, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0026] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation manner of the present disclosure. Other manners that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0027] At the same time, it can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related regulations.

[0028] In the related art, the SQL code is usually evaluated by an exact matching method, that is, the SQL code to be evaluated is matched with the exact SQL code. If the two are inconsistent, it is considered that the SQL code to be evaluated has an error. However, there are some problems with this method. For example, the SQL code to be evaluated has the same semantics as the exact SQL code, but there are only differences in the specific expressions. For example, in the select clause, only the expression order of multiple fields is different, or in the where clause, only the expression order of multiple filtering conditions is different. It may be that the SQL code to be evaluated is correct, but the evaluation result considers that the SQL code is incorrect.

[0029] In view of this, the present disclosure provides a structured query language code evaluation method, device, medium, device and product to improve the accuracy of evaluation and the scalability of the evaluation method.

[0030] Figure 1 is a flowchart of a structured query language code evaluation method shown according to an exemplary embodiment. This method can be applied to an electronic device, such as a terminal device or a server, as Figure 1 shown, this method may include step 11 and step 12.

[0031] In step 11, obtain first query information and a first structured query language SQL code to be evaluated.

[0032] In step 12, a first evaluation result for the first SQL code is determined by at least one agent.

[0033] Exemplarily, the first query information may be text information described in natural language, such as "count the short video play volume in region A". The first query information may also be a query condition including a query dimension and a query metric. For example, the query dimension is region B and the query metric is the number of short video users, and the query condition is used to indicate querying the number of short video users in region B.

[0034] The first SQL code to be evaluated is used to query the answer corresponding to the first query information. Among them, the first SQL code may be generated by an SQL code generation model (such as a Text-to-SQL model) based on the first query information, or may be written by a user according to the first query information. That is, the SQL code evaluation method of the present disclosure can be used to evaluate the SQL code generated by the SQL code generation model, and can also be used to evaluate the SQL code written by the user, and the source of the first SQL code is not limited.

[0035] Among them, an agent is used to determine the first evaluation result according to preset information or a large language model (LLM). The agent can be based on functional modules and models to implement the evaluation of SQL code. Exemplarily, the preset information may include a knowledge base or preset rules. The knowledge base can be pre-constructed, and the evaluation results of the evaluated SQL code can be stored in the knowledge base. The preset rules can be used to indicate the rules that the correct SQL code should satisfy.

[0036] With the above technical solution, a first evaluation result for the first SQL code is determined by at least one agent, and the agent is used to determine the first evaluation result according to preset information or a large language model. In this way, multiple agents can be preset, and the way in which the agent evaluates the SQL code can be set. For example, the agent can evaluate the first SQL code according to preset information, and the agent can evaluate the first SQL code according to a large language model. That is, different agents can evaluate the SQL code in different ways, and the evaluation of SQL code can be based on a multi-agent architecture. Multiple agents work together, which can improve the accuracy of evaluation. Moreover, the number of agents can be set according to the evaluation requirements, and the evaluation methods adopted by the agents can also be set according to the evaluation requirements, improving the scalability of the SQL code evaluation method.

[0037] In one embodiment, at least one agent includes a first agent, a second agent, and / or a third agent. The first agent is used to determine a first evaluation result according to a knowledge base, in which evaluation results of evaluated SQL codes are stored. The second agent is used to determine the first evaluation result according to preset rules, and the preset rules are used to indicate the rules that the correct SQL code should satisfy. The third agent is used to determine the first evaluation result according to a large language model.

[0038] Figure 2 is an architecture diagram of an agent shown according to an exemplary embodiment, as Figure 2 shown, it is an architecture based on multi-agent. The fourth agent can be regarded as a first-level agent, that is, a first-level Agent. The first agent, the second agent, and the third agent are second-level agents, that is, second-level Agents. The first-level agent can be used to manage and allocate the evaluation tasks of multiple second-level agents. The first agent, the second agent, the third agent, and the fourth agent can be integrated into an electronic device.

[0039] Among them, the first agent is used to determine the first evaluation result according to the knowledge base. If the first SQL code is the same as the evaluated SQL code in the knowledge base, the evaluation result can be directly recalled from the knowledge base, improving the evaluation efficiency and accuracy of the first SQL code. The second agent is used to determine the first evaluation result according to the preset rules. Since some error types, such as incorrect use of field types, can be judged according to the rules for using field types, the errors that can be judged according to the preset rules can obtain the first evaluation result through the second agent. The third agent is used to determine the first evaluation result according to a large language model. The large language model refers to a deep learning model trained with a large amount of text data, enabling the model to generate natural language text or understand the meaning of language text, and being able to output error description information in the form of natural language, which can enable users to more intuitively understand the errors and the reasons for the errors in the first SQL code.

[0040] In this way, the knowledge base, the preset rules, and the large language model can be integrated to jointly complete the evaluation of the first SQL code, ensuring the accuracy of the evaluation.

[0041] First, the first implementation manner for the present disclosure to determine the first evaluation result is introduced. Figure 3 is a flowchart of a method for determining a first evaluation result for a first SQL code shown according to an exemplary embodiment, as Figure 3 shown, step 12 may include steps 31 to 35.

[0042] In step 31, through the first agent, it is determined whether there is a second evaluation result in the knowledge base. If it exists, step 32 is executed; if it does not exist, step 33 is executed.

[0043] The implementation manner of step 31 may be as follows: Through the first intelligent agent, determine whether there is an evaluation result in the knowledge base that meets the preset conditions. If so, use the evaluation result that meets the preset conditions as the second evaluation result.

[0044] Among them, the preset conditions include that the annotated SQL code corresponding to the evaluation result is the same as the first SQL code. The preset conditions may also include at least one of the following: the query information corresponding to the evaluation result is the same as the first query information, and the code evaluation information corresponding to the evaluation result is the same as the first code evaluation information, where the first code evaluation information is used to evaluate whether the first SQL code has errors.

[0045] In one embodiment, the preset conditions include that the annotated SQL code corresponding to the evaluation result is the same as the first SQL code. The knowledge base stores the evaluated SQL code and the evaluation result of the evaluated SQL code. For example, it stores SQL code 1 and the evaluation result 1 of SQL code 1. If the first SQL code to be evaluated is the same as this SQL code 1, it can be determined that there is an evaluation result that meets the preset conditions, and the evaluation result 1 that meets the preset conditions can be used as the second evaluation result.

[0046] In one embodiment, the preset conditions include that the annotated SQL code corresponding to the evaluation result is the same as the first SQL code, and the query information corresponding to the evaluation result is the same as the first query information. The knowledge base stores the query information, the evaluated SQL code, and the evaluation result of the evaluated SQL code. The evaluated SQL code is used to query the answer corresponding to this query information. For example, the knowledge base stores query information 1, SQL code 1, and the evaluation result 1 of SQL code 1. If the first query information is the same as query information 1, and the first SQL code to be evaluated is the same as this SQL code 1, it can be determined that there is an evaluation result that meets the preset conditions, and the evaluation result 1 that meets the preset conditions can be used as the second evaluation result.

[0047] In one embodiment, the preset conditions may include that the annotated SQL code corresponding to the evaluation result is the same as the first SQL code, the query information corresponding to the evaluation result is the same as the first query information, and the code evaluation information corresponding to the evaluation result is the same as the first code evaluation information. Query information, evaluated SQL code, code evaluation information, and the evaluation result of the evaluated SQL code may be stored in the knowledge base. By way of example, for instance, query information 1, SQL code 1, code evaluation information 1, and the evaluation result 1 of SQL code 1 are stored in the knowledge base, and the code evaluation information 1 is used to evaluate whether there are errors in SQL code 1. If the first query information is the same as query information 1, the first SQL code to be evaluated is the same as the SQL code 1, and the first code evaluation information is the same as the code evaluation information 1, it may be determined that there is an evaluation result that meets the preset conditions, and the evaluation result 1 that meets the preset conditions may be used as the second evaluation result.

[0048] In addition, the preset conditions may also include that the annotated SQL code corresponding to the evaluation result is the same as the first SQL code, and the code evaluation information corresponding to the evaluation result is the same as the first code evaluation information. The implementation manner of this case may refer to the above embodiment.

[0049] In step 32, the first evaluation result is obtained according to the second evaluation result.

[0050] By way of example, the first evaluation result includes information indicating that the first SQL code is correct, or the first evaluation result includes the code segment with errors in the first SQL code and the error description information for the code segment. The error description information may include the error type and error reason of the code segment described in natural language.

[0051] As an example, the first query information is "Count the list of products where the manufacturer and the channel supplier are in the same city", and the first SQL code is: SELECT * FROM dataset WHERE 'Product manufacturer city' = 'Product channel supplier address' The first evaluation result may be an evaluation result that meets the format requirements. For example, the first evaluation result is: "Label information": "Error", "Error reason": "Should use the product channel supplier city for comparison", "Error type": "Field selection error", "Error annotation": "SELECT * FROM dataset WHERE 'Product manufacturer city' = '''Product channel supplier address''' / *Field selection error, should use the product channel supplier city for comparison.* / " Among them, between ''' and ''', there is a marked code snippet with errors, and between / * and * / are the commented error types and error reasons.

[0052] It should be noted that for the SQL code generated by the generative model, it is generally represented by a common table name. For example, in the above example, the dataset is used to represent the table name.

[0053] In one embodiment, if the format of the second evaluation result meets the requirements, the second evaluation result can be used as the first evaluation result.

[0054] In another embodiment, a first model can be integrated into the first intelligent agent. The first model is, for example, any large language model. If the format of the second evaluation result does not meet the requirements, the first model can be used to output a first evaluation result that meets the format requirements according to the second evaluation result and the specified format requirements. Among them, the format requirements can be: "tag information": "", "error reason": "", "error type": "", "error annotation": "". Specifically, the error annotation is used to indicate that the code snippet with errors is commented between ''' and ''', and the error type and error reason are commented between / * and * / .

[0055] In an application scenario, for example, the SQL code generation model 1 generates SQL code 1 according to the first query information. After evaluating SQL code 1, the evaluation result can be stored in the knowledge base. For example, the first SQL code to be evaluated is generated by the SQL code generation model 2 according to the first query information. Since the capabilities of the SQL code generation model 2 and the SQL code generation model 1 may be similar or the same, the first SQL code generated by the SQL code generation model 2 may be the same as SQL code 1. In this case, the evaluation result can be directly recalled through the knowledge base matching method, without the need for rule judgment or inference by the large language model, ensuring the efficiency and accuracy of the evaluation of the first SQL code.

[0056] In step 33, through the second intelligent agent, it is determined whether the first SQL code meets the preset rules. If not, step 34 is executed; if so, step 35 is executed.

[0057] If the second evaluation result does not exist in the knowledge base, it can be indicated that no evaluated SQL code identical to the first SQL code is matched in the knowledge base. Therefore, the first evaluation result of the first SQL code cannot be obtained from the knowledge base, and the second intelligent agent can be used to determine whether the first SQL code meets the preset rules.

[0058] The implementation manner of step 33 can be: through the second intelligent agent, according to the first query information and the first code evaluation information, it is determined whether the first SQL code meets the preset rules.

[0059] Among them, the first code evaluation information includes the field information of the database tables to be queried to obtain the answers corresponding to the first query information. The field information includes the names, field types, and field parameters of each field in the database tables. Among them, if the first SQL code is generated by an SQL code generation model, such as a Text-to-SQL model, when the Text-to-SQL model generates the first SQL code, it will output which database tables need to be queried. The electronic device can obtain the field information of the pre-stored database tables or can also call the database interface to obtain the field information of the database tables. In addition, the number of database tables to be queried can be one or more. If the number of database tables to be queried is multiple, the first code evaluation information can simultaneously include the field information corresponding to multiple database tables respectively.

[0060] The field information may include the names, field types, and field parameters of each field in the database tables. The field type can represent the type of the value of the field and can be one of the following: integer type, floating-point type, date type, string type. The field parameter refers to the value of the field and can refer to the enumerated value of the field. For example, the enumerated values of the region field may include East China, South China, and Northwest.

[0061] Among them, the field can be the field actually stored in the database table. For example, the name of the field is "number of retained users", and the field type of this field is "int", indicating the integer type, that is, the data of the number of retained users is directly stored in the database table. The field can also refer to the field in the database table that requires aggregation operations to obtain data. For example, the name of the field is "30-day retention rate of new users", and the field type of this field is "float", indicating the floating-point type. The data of the 30-day retention rate of new users is not directly stored in the database table. Through the values of the fields stored in the database table, such as the values of field 1 and field 2, and the calculation method preset for calculating the data of the 30-day retention rate of new users, the data of the 30-day retention rate of new users is aggregated.

[0062] The preset rules include at least one of the following: SQL syntax rules, field type usage rules, index calculation symbol usage rules, the rule that the field names in the first SQL code exist in the field information, and the rule that the field parameters in the first SQL code exist in the field information. According to the preset rules, it can be determined whether the first SQL code has errors of the first specified type. The first specified type includes at least one of the following: SQL syntax error, field type error, index calculation symbol error, field recall error, and field parameter error.

[0063] Among them, the SQL syntax rules can be the syntax rules that the correct SQL code to be preset needs to satisfy, such as punctuation rules, format rules, etc. According to the SQL syntax rules, it can be judged whether there is an SQL syntax error in the first SQL code.

[0064] The field type usage rules include, for example, the rules on which field type values can be calculated or compared. The field type error may refer to the incorrect usage of the field type when calculating or comparing in the first SQL code. For example, in the first SQL code, the value of field A is added to the value of field B, where the field type of field A is string type and the field type of field B is integer type, and the two cannot be directly added, so there is an incorrect usage of the field type.

[0065] The rules for using index calculation symbols include, for example, the rules on whether the mathematical symbols for index calculation in the SQL code match the query intent. The index calculation symbol error refers to the incorrect mathematical symbol for index calculation in the first SQL code. Taking the percentage sign as an example, for example, the query intent of the first query information is to count the percentage, but the percentage sign is not added in the first SQL code, and the resulting value is a decimal.

[0066] The rule that the field names in the first SQL code exist in the field information is used to judge whether the fields referenced in the first SQL code are the fields in the database table. The field recall error refers to the situation where the field names referenced in the first SQL code do not exist in the field information, that is, fields that do not exist in the database table are referenced.

[0067] The rule that the field parameters in the first SQL code exist in the field information is used to judge whether the field parameters in the first SQL code are the parameters in the database table. The field parameter error may refer to the incorrect enumerated value corresponding to the field in the filtering condition in the first SQL code. For example, the filtering condition is Region = Huaxi, but the enumerated values of the Region field in the database table do not include Huaxi.

[0068] In step 34, the first evaluation result is determined according to the rule evaluation result.

[0069] Among them, SQL syntax errors, field type errors, index calculation symbol errors, field recall errors, and field parameter errors, these types of errors can be evaluated through preset rules. A rule judgment module can be integrated in the second intelligent agent to judge whether there are these types of errors in the first SQL code. If there are errors in the first SQL code that do not meet the preset rules, the first evaluation result can be determined according to the rule evaluation result.

[0070] Exemplarily, based on the rule evaluation result, it can be determined which code snippet or snippets in the first SQL code do not meet the preset rules. For example, the rule evaluation result is that the field aaa referenced in the first SQL code does not exist in the field information. The format requirements of the first evaluation result have been described above. In one embodiment, a second model can be integrated in the second agent. The second model can be, for example, any large language model, and the second model can be used to output the first evaluation result that meets the format requirements according to the rule evaluation result and the specified format requirements.

[0071] In step 35, through the third agent, the first evaluation result output by the large language model is obtained.

[0072] Among them, the above-mentioned first specified type of error can be judged by preset rules, while some types of errors cannot be judged by preset rules, such as index calculation logic errors. Therefore, if the first SQL code meets the preset rules, the understanding ability and reasoning ability of the large language model can be further used to judge whether there are other types of errors in the first SQL code. The large language model can be integrated in the third agent.

[0073] The implementation manner of step 35 can be: Through the third agent, according to the prompt template, the first query information, the first SQL code, and the first code evaluation information, a target prompt text is generated, and the target prompt text is input into the large language model to obtain the first evaluation result output by the large language model.

[0074] Among them, the first code evaluation information includes the names of each field in the database table required to obtain the answer. When the first query information is related to time, the first code evaluation information also includes the time when the first query information is submitted. Exemplarily, the first query information is "Count the short video playback volume in area A yesterday". This first query information is related to time, and the time when the query information is submitted is required to judge whether there is a time information error in the first SQL code. For example, if the time when the query information is submitted is March 2nd, it is necessary to judge whether the data counted in the first SQL code is for March 1st. Among them, the time information can refer to absolute time, discrete time, time range, time format, etc. The time information can also include date information, hour information, minute information, etc. In this way, when the query information is related to time, the time when the query information is submitted can be used as a judgment basis to accurately determine whether there is a time information error in the first SQL code.

[0075] The target prompt text is used to guide the large language model to evaluate whether the first SQL code has errors of a specified type. This specified type is called the second specified type, and the second specified type includes at least one of the following: field selection error, index calculation logic error, filtering condition error, time information error.

[0076] Among them, field selection error means that the name of the field referred to in the SQL code is the same as the field name in the field information, but the selected field is incorrect. For example, the first query information is "count the list of products where the manufacturer and the channel supplier are in the same city". The fields referred to in the first SQL code include "city of product manufacturer" and "address of product channel supplier". Although the field "address of product channel supplier" exists in the database table, there is a problem of incorrect field selection. The field that should be referred to is "city of product channel supplier".

[0077] Indicator calculation logic errors can include incorrect indicator calculation methods, incorrect aggregation functions, and incorrect business logic calculations. Among them, incorrect indicator calculation methods refer to logical errors in indicator calculation in the first SQL code. For example, profit needs to be calculated by subtracting cost from revenue, but the first SQL code does not perform subtraction calculations. Incorrect indicator calculation methods can also include incorrect year-on-year calculation logic and incorrect month-on-month calculation logic. Incorrect aggregation functions refer to incorrect aggregation functions used in the first SQL code for calculations. For example, the SUM function should be used for calculations, but the COUNT function is used in the first SQL code. Business logic errors refer to missing, incorrect, or excessive writing of business logic calculations such as grouping, sorting, and deduplication in the first SQL code. For example, according to the query intent of the query information, grouping calculations are not required, but grouping operations are reflected in the first SQL code.

[0078] Filter condition errors mean that the filter conditions do not match the query intent expressed by the query information, such as excessive, incorrect, or missing filter conditions.

[0079] In addition, optionally, the first code evaluation information may further include historical session information and correct SQL code. Among them, if the first query information is information in the form of natural language input by the user on the first page, and before the user inputs the first query information, the user has had one or more rounds of conversations with a certain role on the first page, then the one or more rounds of conversations on the first page before the first query information can be used as historical session information, which can provide reference information for the large language model to accurately determine whether there are errors in the first SQL code. The correct SQL code can be pre-annotated manually and is used to query the code corresponding to the answer of the first query information, serving as reference information for the large language model.

[0080] Exemplarily, the prompt template can be a sentence or a paragraph of text content. The prompt template may include model role information, output format information, output description information, and error type definition. The output format information is used to instruct the large language model to output relevant information in a specified format. The output description information is used to explain the output format information, facilitating the large language model to better understand what information needs to be output. The error type definition may include feature descriptions of the second specified type mentioned above, facilitating the large language model to determine whether the first SQL code has errors of the second specified type.

[0081] As an example, the prompt template is, for example: Model role: You are an SQL code evaluation expert.

[0082] Output format: "Label information": "", "Error reason": "", "Error type": "", "Error annotation": "" Output description: When the label information is correct, it means the first SQL code is correct. When the label information is incorrect, it means the first SQL code has errors. When the label information is incorrect, the error reason is the detailed reason for the error. There may be more than one error, and each error needs to be pointed out. Error type: Select specifically from the error types given in the error type definition. Error annotation: Annotate the first SQL code. Comment the code snippet with errors between ''' and ''', and comment the error type and error reason between / * and * / .

[0083] Error type definition: XXX Field information: {schema} Time of submitting query information: {timeinfo} Query information: {query} First SQL code: {pred_sql} Among them, the specific content of the error type definition is omitted with XXX. {schema} represents the variable of field information, and the field information of the specific database table needs to be filled in here. {timeinfo} represents the variable of the time when the query information is submitted, and the specific time when the query information is submitted needs to be filled in here. {query} represents the variable of the query information, and the specific first query information needs to be filled in here. {pred_sql} represents the variable of the first SQL code, and the first SQL code to be evaluated needs to be filled in here. Filling the first query information, the first code evaluation information, and the first SQL code into the corresponding positions in the prompt template can obtain the target prompt text (prompt), and this target prompt text is used as the input of the large language model, and the large language model can output the corresponding evaluation result according to the target prompt text.

[0084] Exemplarily, the first query information is "count the list of products where the manufacturer and the channel supplier are in the same city", and the information output by the large language model can refer to the example of the first evaluation result in the explanation of step 32.

[0085] Through the above solution, in this embodiment, the priorities of the first intelligent agent, the second intelligent agent, and the third intelligent agent can be preset. For example, the priority of the first intelligent agent is the highest, the priority of the second intelligent agent is the second, and the priority of the third intelligent agent is the lowest. That is, first, the first intelligent agent determines whether there is an evaluation result in the knowledge base that meets the preset conditions. If so, the evaluation result can be directly recalled from the knowledge base without rule judgment or inference by the large language model. If no evaluation result that meets the preset conditions is matched from the knowledge base, the second intelligent agent can then be used to determine whether the first SQL code has errors that do not meet the preset rules. Since the types of errors that can be judged according to the preset rules are limited, if the first SQL code meets the preset rules, the third intelligent agent can then be used to obtain the first evaluation result output by the large language model. The first large language model can be used to determine whether the first SQL code has a specified type of error, and this specified type of error can be an error that cannot be judged according to the preset rules, and the understanding ability and reasoning ability of the large language model can be further used for judgment. The large language model can generate error information described in natural language to enable users to more intuitively understand the errors existing in the first SQL code.

[0086] Therefore, in the process of determining the first evaluation result, at least one intelligent agent can include one of the first intelligent agent, the second intelligent agent, and the third intelligent agent, can also include two of these three intelligent agents, or can also include all three of these intelligent agents at the same time.

[0087] Exemplarily, the evaluation of some types of errors is more difficult, such as the error in the month-on-month calculation logic and the error in the year-on-year calculation logic. The evaluation results of SQL codes with these error types can be stored in advance and built into a knowledge base. Therefore, for SQL codes with greater evaluation difficulty, the corresponding evaluation results can be retrieved from the knowledge base as much as possible, without the need for rule judgment or inference by a large language model, ensuring the accuracy and efficiency of the evaluation.

[0088] In addition, for example, the fourth intelligent agent can manage and allocate evaluation tasks according to the priorities of the first, second, and third intelligent agents. The fourth intelligent agent can store the priority information of the first, second, and third intelligent agents set in advance. For example, the fourth intelligent agent first allocates the evaluation task to the first intelligent agent. If it receives the information that the second evaluation result does not exist in the knowledge base returned by the first intelligent agent, it then allocates the evaluation task to the second intelligent agent.

[0089] The second implementation manner for the present disclosure to determine the first evaluation result is introduced below.

[0090] Figure 4 It is a flowchart of a method for determining the first evaluation result of the first SQL code shown according to another exemplary embodiment, as Figure 4 shown. Step 12 may include steps 41 to 43.

[0091] In step 41, according to the first query information and the first SQL code, the target error type is determined.

[0092] In step 42, according to the preset correspondence between the error type and the intelligent agent, and the target error type, the target intelligent agent corresponding to the target error type is determined.

[0093] Among them, the target error type is the type of error predicted to exist in the first SQL code. Exemplarily, the fourth intelligent agent can predict the target error type and determine the corresponding target intelligent agent. A prediction model can be integrated in the fourth intelligent agent. The prediction model can be any large language model. The prediction model can be used to determine the target error type according to the first query information, the first SQL code, and the first code evaluation information. For example, the prediction model can output the error type most likely to exist in the predicted first SQL code as the target error type, or the prediction model can output the probability of the first SQL code having various types of errors, and take the type corresponding to the maximum probability as the target error type.

[0094] The target agent is one of at least one agent. Among them, the preset correspondence between error types and agents can be set in advance, and this preset correspondence can be in the form of a table or the like, for example. Exemplarily, the preset correspondence can be established according to the evaluation methods of each agent and the evaluation difficulty of error types. For example, for error types with relatively high evaluation difficulty, they can correspond to the first agent; for error types that can be judged according to preset rules (such as the above-mentioned first specified type), they can correspond to the second agent; for error types that cannot be judged by preset rules (such as the above-mentioned second specified type), they can correspond to the third agent.

[0095] In step 43, the first evaluation result is determined by the target agent.

[0096] In one embodiment, the implementation manner of step 43 can be: If the target agent is the first agent, then the first agent is used to determine whether there is a second evaluation result in the knowledge base. If there is a second evaluation result in the knowledge base, then the first evaluation result is obtained according to the second evaluation result, where the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; If there is no second evaluation result in the knowledge base, or the target agent is the second agent, then the second agent is used to determine whether the first SQL code meets the preset rules. If the first SQL code does not meet the preset rules, then the first evaluation result is determined according to the rule evaluation result; If the first SQL code meets the preset rules, or the target agent is the third agent, then the first evaluation result output by the large language model is obtained through the third agent.

[0097] Among them, the implementation manner of determining whether there is a second evaluation result in the knowledge base through the first agent can refer to the explanation of step 31. The implementation manner of obtaining the first evaluation result according to the second evaluation result can refer to the explanation of step 32. The implementation manner of determining whether the first SQL code meets the preset rules through the second agent can refer to the explanation of step 33. The implementation manner of determining the first evaluation result according to the rule evaluation result can refer to the explanation of step 34. The implementation manner of obtaining the first evaluation result output by the large language model through the third agent can refer to the explanation of step 35.

[0098] Exemplarily, the target agent is determined according to the predicted target error type and can be one of the first agent, the second agent, and the third agent. That is, in this embodiment, the target agent can be first determined according to the predicted target error type. The target agent first performs the evaluation task, and then the first evaluation result can be obtained based on the evaluation result of the agent and the priority of the agent, which can improve the evaluation efficiency. In this embodiment, the fourth agent can store the priority information of the first agent, the second agent, and the third agent set in advance. For example, after the fourth agent determines the target agent, if the target agent is the second agent, the evaluation task is first assigned to the second agent. If the information that the first SQL code returned by the second agent meets the preset rules is received, the evaluation task is then assigned to the third agent.

[0099] Based on the same inventive concept, the present disclosure also provides a structured query language code evaluation device. Figure 5 It is a block diagram of a structured query language code evaluation device shown according to an exemplary embodiment, as Figure 5 shown. The device 50 may include: An acquisition module 51, configured to acquire first query information and a first structured query language SQL code to be evaluated, where the first SQL code is used to query an answer corresponding to the first query information; A determination module 52, configured to determine a first evaluation result for the first SQL code through at least one agent; wherein, the agent is configured to determine the first evaluation result according to preset information or a large language model.

[0100] Optionally, the preset information includes a knowledge base or preset rules, the at least one agent includes a first agent, a second agent, and / or a third agent. The first agent is configured to determine the first evaluation result according to the knowledge base, and the evaluation results of the evaluated SQL codes are stored in the knowledge base. The second agent is configured to determine the first evaluation result according to the preset rules, and the preset rules are used to indicate the rules that the correct SQL code should satisfy. The third agent is configured to determine the first evaluation result according to the large language model.

[0101] Optionally, the determination module 52 includes: A first determination sub-module, configured to determine, through the first agent, whether there is a second evaluation result in the knowledge base. If there is the second evaluation result in the knowledge base, the first evaluation result is obtained according to the second evaluation result, where the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; A second determination sub-module, configured to, if the second evaluation result does not exist in the knowledge base, determine, through the second agent, whether the first SQL code meets the preset rules, and if the first SQL code does not meet the preset rules, determine the first evaluation result according to the rule evaluation result; A third determination sub-module, configured to, if the first SQL code meets the preset rules, obtain, through the third agent, the first evaluation result output by the large language model.

[0102] Optionally, the determination module 52 includes: A type determination sub-module, configured to determine a target error type according to the first query information and the first SQL code, where the target error type is the type of error predicted to exist in the first SQL code; An agent determination sub-module, configured to determine a target agent corresponding to the target error type according to a preset correspondence between error types and agents and the target error type, where the target agent is one of the at least one agent; A result determination sub-module, configured to determine the first evaluation result through the target agent.

[0103] Optionally, the result determination sub-module includes: A first determination sub-module, configured to, if the target agent is the first agent, determine, through the first agent, whether a second evaluation result exists in the knowledge base, and if the second evaluation result exists in the knowledge base, obtain the first evaluation result according to the second evaluation result, where the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; A second determination sub-module, configured to, if the second evaluation result does not exist in the knowledge base, or the target agent is the second agent, determine, through the second agent, whether the first SQL code meets the preset rules, and if the first SQL code does not meet the preset rules, determine the first evaluation result according to the rule evaluation result; A third determination sub-module, configured to, if the first SQL code meets the preset rules, or the target agent is the third agent, obtain, through the third agent, the first evaluation result output by the large language model.

[0104] Optionally, the first determination sub-module is configured to: Determine, through the first agent, whether there is an evaluation result that meets the preset conditions in the knowledge base, and if so, use the evaluation result that meets the preset conditions as the second evaluation result; Among them, the preset conditions include that the labeled SQL code corresponding to the evaluation result is the same as the first SQL code, and the preset conditions also include at least one of the following: the query information corresponding to the evaluation result is the same as the first query information, the code evaluation information corresponding to the evaluation result is the same as the first code evaluation information, where the first code evaluation information is used to evaluate whether the first SQL code has errors.

[0105] Optionally, the second determination sub-module is used to: Through the second intelligent agent, determine whether the first SQL code meets the preset rules according to the first query information and the first code evaluation information; Among them, the first code evaluation information includes the field information of the database table to be queried to obtain the answer, and the field information includes the names, field types, and field parameters of each field in the database table; The preset rules include at least one of the following: SQL syntax rules, field type usage rules, metric calculation symbol usage rules, the rule that the field names in the first SQL code exist in the field information, the rule that the field parameters in the first SQL code exist in the field information.

[0106] Optionally, the third determination sub-module is used to: Through the third intelligent agent, generate a target prompt text according to the prompt word template, the first query information, the first SQL code, and the first code evaluation information, and input the target prompt text into the large language model to obtain the first evaluation result output by the large language model; Among them, the first code evaluation information includes the names of each field in the database table to be queried to obtain the answer, and when the first query information is related to time, the first code evaluation information also includes the time when the first query information is submitted; The target prompt text is used to guide the large language model to evaluate whether the first SQL code has errors of a specified type, and the specified type includes at least one of the following: field selection error, metric calculation logic error, filtering condition error, time information error.

[0107] Optionally, the first evaluation result includes information indicating that the first SQL code is correct, or the first evaluation result includes the code segment with errors in the first SQL code and the error description information for the code segment.

[0108] Next, refer to Figure 6, which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0109] As Figure 6 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0110] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 shows the electronic device 600 having various devices, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be implemented or had alternatively.

[0111] Particularly, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

[0112] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0113] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0114] The above computer-readable medium can be included in the above electronic device; it can also exist separately without being assembled into the electronic device.

[0115] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: obtain first query information and a first Structured Query Language (SQL) code to be evaluated, where the first SQL code is used to query an answer corresponding to the first query information. Determine a first evaluation result for the first SQL code through at least one agent; wherein, the agent is used to determine the first evaluation result according to preset information or a large language model.

[0116] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0118] The modules described in the embodiments of the present disclosure may be implemented in software or in hardware. Wherein, the name of the module does not constitute a limitation to the module itself in some cases. For example, the determination module may also be described as "the module for determining the first evaluation result".

[0119] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0120] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash Memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] According to one or more embodiments of the present disclosure, Example 1 provides a method for evaluating Structured Query Language code, the method comprising: Obtaining first query information and a first Structured Query Language (SQL) code to be evaluated, the first SQL code being used to query an answer corresponding to the first query information; Determining a first evaluation result for the first SQL code through at least one agent; wherein the agent is used to determine the first evaluation result according to preset information or a large language model.

[0122] According to one or more embodiments of the present disclosure, Example 2 provides that in the method of Example 1, the preset information includes a knowledge base or preset rules, the at least one agent includes a first agent, a second agent, and / or a third agent, the first agent is used to determine the first evaluation result according to the knowledge base, the knowledge base stores evaluation results of evaluated SQL codes, the second agent is used to determine the first evaluation result according to the preset rules, the preset rules are used to indicate the rules that a correct SQL code should satisfy, and the third agent is used to determine the first evaluation result according to a large language model.

[0123] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2. Determining a first evaluation result for the first SQL code through at least one agent includes: Determine, by the first agent, whether a second evaluation result exists in the knowledge base. If the second evaluation result exists in the knowledge base, obtain the first evaluation result according to the second evaluation result, where the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; If the second evaluation result does not exist in the knowledge base, determine, by the second agent, whether the first SQL code meets the preset rules. If the first SQL code does not meet the preset rules, determine the first evaluation result according to the rule evaluation result; If the first SQL code meets the preset rules, obtain the first evaluation result output by the large language model through the third agent.

[0124] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 2. Determining a first evaluation result for the first SQL code through at least one agent includes: Determine a target error type according to the first query information and the first SQL code, where the target error type is the type of error predicted to exist in the first SQL code; Determine a target agent corresponding to the target error type according to a preset correspondence relationship between error types and agents and the target error type, where the target agent is one of the at least one agent; Determine the first evaluation result through the target agent.

[0125] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4. Determining the first evaluation result through the target agent includes: If the target agent is the first agent, determine, by the first agent, whether a second evaluation result exists in the knowledge base. If the second evaluation result exists in the knowledge base, obtain the first evaluation result according to the second evaluation result, where the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; If the second evaluation result does not exist in the knowledge base, or the target agent is the second agent, determine, by the second agent, whether the first SQL code meets the preset rules. If the first SQL code does not meet the preset rules, determine the first evaluation result according to the rule evaluation result; If the first SQL code meets the preset rules, or the target agent is the third agent, then through the third agent, obtain the first evaluation result output by the large language model.

[0126] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 3 or Example 5. The determining, by the first agent, whether there is a second evaluation result in the knowledge base includes: Determine, by the first agent, whether there is an evaluation result in the knowledge base that meets the preset conditions. If so, use the evaluation result that meets the preset conditions as the second evaluation result; Wherein, the preset conditions include that the labeled SQL code corresponding to the evaluation result is the same as the first SQL code, and the preset conditions further include at least one of the following: the query information corresponding to the evaluation result is the same as the first query information, the code evaluation information corresponding to the evaluation result is the same as the first code evaluation information, wherein the first code evaluation information is used to evaluate whether there is an error in the first SQL code.

[0127] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 3 or Example 5. The determining, by the second agent, whether the first SQL code meets the preset rules includes: Determine, by the second agent, whether the first SQL code meets the preset rules according to the first query information and the first code evaluation information; Wherein, the first code evaluation information includes the field information of the database tables required to obtain the answer, and the field information includes the names, field types, and field parameters of each field in the database tables; The preset rules include at least one of the following: SQL syntax rules, field type usage rules, index calculation symbol usage rules, the rule that the field names in the first SQL code exist in the field information, the rule that the field parameters in the first SQL code exist in the field information.

[0128] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 3 or Example 5. The obtaining, by the third agent, of the first evaluation result output by the large language model includes: Generate, by the third agent, a target prompt text according to the prompt word template, the first query information, the first SQL code, and the first code evaluation information, and input the target prompt text into the large language model to obtain the first evaluation result output by the large language model; Among them, the first code evaluation information includes the names of each field in the database table to be queried to obtain the answer. When the first query information is related to time, the first code evaluation information further includes the time when the first query information is submitted; The target prompt text is used to guide the large language model to evaluate whether the first SQL code has errors of a specified type, and the specified type includes at least one of the following: field selection error, metric calculation logic error, filtering condition error, time information error.

[0129] According to one or more embodiments of the present disclosure, Example 9 provides the method of Example 1. The first evaluation result includes information indicating that the first SQL code is correct, or the first evaluation result includes the code segment with errors in the first SQL code and error description information for the code segment.

[0130] According to one or more embodiments of the present disclosure, Example 10 provides a structured query language code evaluation device, and the device includes: An acquisition module, configured to acquire first query information and a first structured query language (SQL) code to be evaluated, where the first SQL code is used to query the answer corresponding to the first query information; A determination module, configured to determine a first evaluation result of the first SQL code through at least one agent; where the agent is configured to determine the first evaluation result according to preset information or a large language model.

[0131] According to one or more embodiments of the present disclosure, Example 11 provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the method described in any one of Examples 1 to 9 are implemented.

[0132] According to one or more embodiments of the present disclosure, Example 12 provides an electronic device, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method described in any one of Examples 1 to 9.

[0133] According to one or more embodiments of the present disclosure, Example 13 provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of Examples 1 to 9 are implemented.

[0134] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0135] In addition, although the operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0136] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

Claims

1. A structured query language code evaluation method, characterized in that: The method comprises: Acquire first query information and a first structured query language SQL code to be evaluated, where the first SQL code is used to query an answer corresponding to the first query information; A first evaluation result of the first SQL code is determined by at least one intelligent agent; wherein the intelligent agent is used to determine the first evaluation result according to preset information or a large language model.

2. The method according to claim 1, characterized in that The preset information includes a knowledge base or preset rules, the at least one agent includes a first agent, a second agent and / or a third agent, the first agent is used to determine the first evaluation result based on the knowledge base, the knowledge base stores the evaluation results of the evaluated SQL code, the second agent is used to determine the first evaluation result based on the preset rules, the preset rules are used to indicate the rules that the correct SQL code should satisfy, and the third agent is used to determine the first evaluation result based on a large language model.

3. The method according to claim 2, characterized in that The determining, by at least one intelligent agent, a first evaluation result of the first SQL code comprises: Determine, by the first agent, whether there is a second evaluation result in the knowledge base, and if the second evaluation result exists in the knowledge base, obtain the first evaluation result according to the second evaluation result, wherein the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; If the second evaluation result does not exist in the knowledge base, determining whether the first SQL code satisfies the preset rule through the second agent, and if the first SQL code does not satisfy the preset rule, determining the first evaluation result according to the rule evaluation result; If the first SQL code satisfies the preset rule, the first evaluation result output by the large language model is obtained through the third agent.

4. The method according to claim 2, characterized in that: The determining, by at least one intelligent agent, a first evaluation result of the first SQL code comprises: Determine a target error type according to the first query information and the first SQL code, where the target error type is a type of error predicted to exist in the first SQL code; According to a preset correspondence between an error type and an agent, and the target error type, determining a target agent corresponding to the target error type, the target agent being one of the at least one agent; The first evaluation result is determined through the target agent.

5. The method according to claim 4, characterized in that The step of determining the first evaluation result by the target agent includes: If the target agent is the first agent, determining whether a second evaluation result exists in the knowledge base through the first agent, and if the second evaluation result exists in the knowledge base, obtaining the first evaluation result according to the second evaluation result, wherein the evaluated SQL code corresponding to the second evaluation result is the same as the first SQL code; If the second evaluation result does not exist in the knowledge base, or the target agent is the second agent, determining whether the first SQL code satisfies the preset rule through the second agent; if the first SQL code does not satisfy the preset rule, determining the first evaluation result according to the rule evaluation result; If the first SQL code satisfies the preset rule, or the target agent is the third agent, the first evaluation result output by the large language model is obtained through the third agent.

6. The method according to claim 3 or 5, characterized in that: The determining, by the first agent, whether there is a second evaluation result in the knowledge base includes: Determine, by means of the first agent, whether there is an evaluation result satisfying a preset condition in the knowledge base, and if so, use the evaluation result satisfying the preset condition as the second evaluation result; Among them, the preset condition includes that the marked SQL code corresponding to the evaluation result is the same as the first SQL code, and the preset condition also includes at least one of the following: the query information corresponding to the evaluation result is the same as the first query information, and the code evaluation information corresponding to the evaluation result is the same as the first code evaluation information, wherein the first code evaluation information is used to evaluate whether there is an error in the first SQL code.

7. The method according to claim 3 or 5, characterized in that: The determining, by the second agent, whether the first SQL code satisfies the preset rule comprises: Determining, by the second agent, whether the first SQL code satisfies the preset rule according to the first query information and the first code evaluation information; The first code evaluation information includes field information of a database table that needs to be queried to obtain the answer, and the field information includes the name, field type and field parameter of each field in the database table; The preset rules include at least one of the following: SQL syntax rules, field type usage rules, indicator calculation symbol usage rules, rules that the field name in the first SQL code exists in the field information, and rules that the field parameters in the first SQL code exist in the field information.

8. The method according to claim 3 or 5, characterized in that: The obtaining, by the third agent, the first evaluation result output by the large language model comprises: Generate a target prompt text by the third agent according to the prompt word template, the first query information, the first SQL code and the first code evaluation information, and input the target prompt text into the large language model to obtain the first evaluation result output by the large language model; The first code evaluation information includes the names of the fields in the database table that need to be queried to obtain the answer, and when the first query information is related to time, the first code evaluation information also includes the time when the first query information is submitted; The target prompt text is used to guide the large language model to evaluate whether the first SQL code has a specified type of error, and the specified type includes at least one of the following: field selection error, indicator calculation logic error, screening condition error, and time information error.

9. The method according to claim 1, characterized in that: The first evaluation result includes information indicating that the first SQL code is correct, or the first evaluation result includes a code snippet with an error in the first SQL code and error description information for the code snippet.

10. A structured query language code evaluation device, characterized in that: The device comprises: An acquisition module, used to acquire first query information and a first structured query language SQL code to be evaluated, wherein the first SQL code is used to query an answer corresponding to the first query information; A determination module is used to determine a first evaluation result of the first SQL code through at least one intelligent agent; wherein the intelligent agent is used to determine the first evaluation result based on preset information or a large language model.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method described in any one of claims 1 to 9 are implemented.

12. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images