Structured query language evaluation method and device, equipment and medium

Abstract

The invention provides a structured query language evaluation method and device, equipment and a medium, and relates to the technical field of data processing, in particular to the technical field of large model and natural language processing. According to the specific implementation scheme, a query problem is obtained; generating a prediction structured query language according to the query problem by adopting a large language model; if the target structured query language exists, evaluating the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language to obtain an evaluation result corresponding to the predicted structured query language; if the target structured query language does not exist, the accuracy of the predicted structured query language is evaluated according to the semantic analysis result of the predicted structured query language, and an evaluation result corresponding to the predicted structured query language is obtained, and the accuracy of predicted structured query language evaluation is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, specifically to the field of large models and natural language processing technology, and especially to a structured query language evaluation method, device, equipment and medium. Background Art

[0002] With the continuous advancement of database technology, SQL (Structured Query Language) has become an indispensable tool for data processing and analysis. However, writing effective SQL requires not only solid logical thinking skills, but also a deep understanding of database operations. Faced with the growing and complex business needs, ensuring the correctness of SQL, the accuracy of semantics, and the high efficiency of execution is a major challenge for developers.

[0003] It can be seen that providing a method to effectively evaluate SQL accuracy is of great significance for improving development efficiency and ensuring data processing quality. Summary of the invention

[0004] The present disclosure provides a structured query language evaluation method, apparatus, device and medium.

[0005] According to one aspect of the present disclosure, a structured query language evaluation method is provided, the method comprising:

[0006] Get the query question entered by the user;

[0007] Using a large language model to generate a predictive structured query language corresponding to the query question according to the query question;

[0008] If there is a target structured query language corresponding to the query question, then based on the predicted structured query language and the target structured query language, the accuracy of the predicted structured query language is evaluated to obtain an evaluation result corresponding to the predicted structured query language;

[0009] If there is no target structured query language corresponding to the query question, the accuracy of the predicted structured query language is evaluated according to the semantic analysis result of the predicted structured query language to obtain an evaluation result corresponding to the predicted structured query language.

[0010] According to another aspect of the present disclosure, a structured query language evaluation device is provided, the device comprising:

[0011] An acquisition module is used to obtain the query question input by the user;

[0012] A generation module, configured to generate a predictive structured query language corresponding to the query question according to the query question using a large language model;

[0013] A first evaluation module is used to evaluate the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language if there is a target structured query language corresponding to the query question, so as to obtain an evaluation result corresponding to the predicted structured query language;

[0014] The second evaluation module is used to evaluate the accuracy of the predicted structured query language according to the semantic analysis result of the predicted structured query language if there is no target structured query language corresponding to the query question, so as to obtain an evaluation result corresponding to the predicted structured query language.

[0015] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method in any of the above technical solutions.

[0019] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the above technical solutions.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements any one of the methods described in the above technical solutions when executed by a processor.

[0021] The present disclosure provides a structured query language evaluation method, apparatus, device and storage medium. The present disclosure receives query questions input by users and uses a large language model to generate corresponding predictive structured query languages, thereby significantly improving the conversion efficiency from natural language to structured query languages. Secondly, in the presence of a target structured query language, the present disclosure evaluates the accuracy of the predicted statement by comparing the predicted structured query language and the target structured query language. In the absence of a target structured query language, the accuracy of the predicted statement is evaluated through in-depth semantic analysis. It can be seen that the scheme of the present disclosure can effectively evaluate the generated predictive structured query language regardless of whether there is a target structured query language as a reference, thereby improving the accuracy of the predictive structured query language evaluation.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a schematic diagram of the steps of the structured query language evaluation method in an embodiment of the present disclosure;

[0025] Figure 2 is a flow chart corresponding to the structured query language evaluation method in the embodiment of the present disclosure;

[0026] Figure 3 A functional block diagram of a structured query language evaluation device in an embodiment of the present disclosure;

[0027] Figure 4 The block diagram is a block diagram of an electronic device for implementing the structured query language evaluation method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] In related technical solutions, the following methods are often used to evaluate SQL:

[0030] The first method is to first generate SQL by evaluating the model, and then compare the matching degree between the model-generated SQL and the standard SQL to judge its accuracy. This evaluation method has the potential problem of underestimation. For example, in actual applications, although the generated SQL execution result is correct, if the generated SQL does not completely match the standard SQL at the string level, it may only be that the order of the SELECT columns is different, or the SQL with the same function but different expression is used, it may be considered that there is a problem with the query, which will lead to misjudgment.

[0031] The second method is: after generating SQL, first convert the generated SQL and the standard SQL into standard form, and then calculate their matching degree, so as to obtain the evaluation result. This evaluation method effectively solves the problem of misjudgment caused by different column orders by standardizing expressions. In addition, this method can also reduce misjudgment caused by format differences by sorting elements such as columns and tables in SQL queries and normalizing the aliases used in SQL. Although this method can effectively solve the misjudgment problem caused by the first method to a certain extent, it cannot handle more complex SQL semantic changes.

[0032] The third method is to evaluate the accuracy of SQL queries by calculating the proportion of the number of correct SQL execution results in the entire data set, that is, if the result returned by a SQL query is consistent with the expected result, the query is considered accurate. This method only requires that the result finally returned by the database execution engine is consistent with the result executed by the correct SQL, but it cannot guarantee that the generated SQL is semantically correct, so it will lead to an inflated accuracy rate.

[0033] In addition, in enterprise application scenarios, there are often only query problems, and it is difficult to obtain matching standard SQL on a large scale. In this case, traditional evaluation methods cannot fully play their role.

[0034] In order to solve the above problems, the present disclosure provides a structured query language evaluation method, see Figure 1 As shown, Figure 1 : is a schematic diagram of the steps of the structured query language evaluation method in an embodiment of the present disclosure, the method comprising:

[0035] Step S101, obtaining a query question input by a user.

[0036] Specifically, obtaining the query questions input by the user is the first step in the data processing flow, which involves receiving and understanding the data retrieval requests made by the user in the form of natural language. In this process, the system first needs to be able to receive the user's input, which can be text, voice or other forms of query instructions. Subsequently, natural language processing (NLP) technology is used to parse and understand the intent and content of these queries, including identifying key information such as operation objects, condition restrictions, etc. The step of obtaining the query questions input by the user can form the basis for effective communication between the user and the database, ensuring that subsequent steps, such as using a large language model to generate the corresponding SQL, can accurately reflect the user's query needs.

[0037] Step S102: using a large language model to generate a predicted structured query language corresponding to the query question according to the query question.

[0038] Specifically, after obtaining the query question input by the user, a large language model is used to generate a predictive structured query language (also called predictive SQL) corresponding to the query question. When the user makes a query request in natural language, the large language model first parses the semantic content of these queries and identifies key information, such as the data fields, conditions, and data tables that need to be retrieved. Then, the large language model uses its deep learning algorithm to convert these natural language descriptions into precise SQL. These SQLs are structured, which means that they follow strict grammatical rules and can be executed directly in the database management system to retrieve or manipulate data. In this way, by using a large language model, human language can be automatically translated into code that can be understood and executed by machines, greatly improving the efficiency of data processing and the convenience of users. For example, the large language model can be a large model such as Wenxin Yiyan.

[0039] Step S103: if there is a target structured query language corresponding to the query question, the accuracy of the predicted structured query language is evaluated according to the predicted structured query language and the target structured query language to obtain an evaluation result corresponding to the predicted structured query language.

[0040] Specifically, it should be noted that for the target structured query language (also called target SQL), when obtaining the query question input by the user, it can be known whether the query question has a corresponding target structured query language, that is, the target SQL is usually a pre-defined statement that is known to accurately reflect the user's query requirements.

[0041] If there is a predefined target structured query language (SQL) statement that directly corresponds to the user query question, the accuracy of the predicted statement is evaluated by comparing the predicted SQL generated by the large language model with the target SQL. This comparison process usually involves a detailed comparative analysis of the structure, grammar, and logic of the two to determine whether the predicted statement can accurately reflect the intention and needs of the original query, so that the corresponding evaluation result can be obtained. The evaluation results are usually presented in the form of scores, percentages, or other forms of indicators, which indicate the degree of match between the predicted SQL and the target statement, thereby providing the system with direct feedback on the quality of its generated statements. It can be seen that the embodiment of the present application provides a clear and direct evaluation mechanism, so that the accuracy of the predicted SQL can be effectively verified, thereby improving the accuracy of SQL evaluation.

[0042] Step S104: if there is no target structured query language corresponding to the query question, the accuracy of the predicted structured query language is evaluated according to the semantic analysis result of the predicted structured query language to obtain an evaluation result corresponding to the predicted structured query language.

[0043] Specifically, in the absence of a target structured query language (SQL) statement that directly corresponds to the query question, the accuracy of the predicted SQL is evaluated through in-depth semantic analysis. Semantic analysis is a technology for understanding the meaning of text, which can identify and interpret keywords, operations, and logical relationships in SQL. In this process, natural language processing technology is first used to parse the semantic content in the predicted SQL, and then these statements are evaluated based on the preset semantic rules and database schema to see whether they logically conform to the user's original query intent, and finally the corresponding evaluation results can be obtained. This method allows for effective accuracy evaluation of the generated SQL in the absence of a clear reference standard, ensuring that they can be correctly executed in actual applications.

[0044] The present disclosure provides a structured query language evaluation method, apparatus, device and storage medium. The present disclosure receives query questions input by users and uses a large language model to generate corresponding predictive structured query languages, thereby significantly improving the conversion efficiency from natural language to structured query languages. Secondly, in the presence of a target structured query language, the present disclosure evaluates the accuracy of the predicted statement by comparing the predicted structured query language and the target structured query language. In the absence of a target structured query language, the accuracy of the predicted statement is evaluated through in-depth semantic analysis. It can be seen that the scheme of the present disclosure can effectively evaluate the generated predictive structured query language regardless of whether there is a target structured query language as a reference, thereby improving the accuracy of the predictive structured query language evaluation.

[0045] In some optional embodiments, the accuracy of the predicted structured query language is evaluated according to the predicted structured query language and the target structured query language to obtain an evaluation result corresponding to the predicted structured query language, including:

[0046] Obtaining a first execution result corresponding to the predicted structured query language;

[0047] Acquire a second execution result corresponding to the target structured query language;

[0048] The accuracy of the predicted structured query language is evaluated according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language.

[0049] Specifically, in the case where there is a target structured query language corresponding to the query question, when evaluating the accuracy of the predicted structured query language, the predicted SQL is first executed to obtain a first execution result, and then the target SQL is executed to obtain a second execution result.

[0050] Next, the first execution result and the second execution result are compared to analyze the similarities and differences between them. This comparison may include checking the number of rows of data returned, the data content, the data order, and any potential anomalies or errors. Based on these comparison results, the accuracy of the predicted SQL can be evaluated, and an evaluation result can be obtained. For the evaluation result, this may include a score, rating, or other form of feedback that clearly indicates the degree of match between the predicted statement and the target statement. This evaluation result is crucial for optimizing the language model, improving the accuracy of SQL generation, and ultimately improving user satisfaction.

[0051] In this way, by obtaining the execution results of the predicted SQL and the target SQL and carefully comparing the two results, the accuracy of the predicted statement can be quantified, which is further conducive to timely discovery and correction of possible deviations.

[0052] In some optional embodiments, the accuracy of the predicted structured query language is evaluated according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language, including:

[0053] Performing a grammatical comparison between the predicted structured query language and the target structured query language to obtain a first comparison result;

[0054] Compare the first execution result and the second execution result for consistency, to obtain a second comparison result;

[0055] Obtaining a first preliminary evaluation result according to the first comparison result and the second comparison result;

[0056] An evaluation result corresponding to the predicted structured query language is obtained according to the first preliminary evaluation result.

[0057] Specifically, the predicted SQL is first compared with a predefined and accurate target SQL for syntax comparison. This step aims to check the consistency of the two in terms of structure and syntax rules, and generate the first comparison result. Next, the two statements are executed and their first and second execution results are obtained. The two sets of results are then compared to verify whether they return the same or logically equivalent data, thereby obtaining the second comparison result. Combining the first and second comparison results, the system generates a preliminary evaluation result that combines the grammatical correctness and consistency of the execution results. Finally, based on this preliminary evaluation result, the system further analyzes and obtains the final evaluation result, providing quantitative feedback for the accuracy of the predicted SQL. This process ensures that the generated SQL is not only correct in form, but also accurately executes the user's query intent in terms of function, thereby improving the reliability and effectiveness of the automatic SQL generation system.

[0058] In this way, the correctness of the statement structure is first ensured by comparing the syntax of the predicted SQL with the target SQL, and then the logical correctness and data accuracy of the statement are verified by comparing the results. Through these two steps of comparison, a preliminary evaluation result can be generated, and then the final evaluation result can be obtained, which not only improves the reliability of the predicted SQL, but also enhances the accuracy of the entire system and the trust of users, ensuring the efficiency of the data processing process and the accurate satisfaction of user query needs. That is, by comprehensively comparing the syntax consistency of the predicted SQL and the target SQL and the data consistency of their respective execution results, a comprehensive evaluation of the accuracy of the predicted SQL is achieved.

[0059] In some optional embodiments, obtaining an evaluation result corresponding to the predicted structured query language according to the first preliminary evaluation result includes:

[0060] Obtain the number of correct evaluations in the first preliminary evaluation result;

[0061] The ratio between the number of correct evaluations and the total number of first preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

[0062] Specifically, after obtaining the first preliminary evaluation result, since the first preliminary evaluation result includes the results of correct evaluation and the results of incorrect evaluation, the embodiment of the present disclosure needs to count the number of correct evaluations from the preliminary evaluation results, that is, the number of cases where the predicted SQL completely matches the target SQL in terms of syntax and execution results. Subsequently, the number of correct evaluations is compared with the total number of preliminary evaluation results to calculate a ratio. This ratio reflects the percentage of correctness of the predicted SQL in all evaluation items, thereby providing an intuitive and quantifiable evaluation result to measure the overall accuracy of the predicted statement. This method makes the evaluation results more objective and accurate, helps to further optimize the language model, and improves the accuracy of SQL generation.

[0063] In this way, by extracting the number of correct evaluations from the preliminary evaluation results and calculating the ratio between this number and the total number of evaluations, a quantitative evaluation of the accuracy of the predicted SQL can be achieved. This method not only provides a clear indicator to measure the accuracy of the predicted statement, but also through the ratio calculation, it can objectively reflect the overall performance of the predicted statement in all evaluation items. In addition, such a quantitative evaluation mechanism helps to quickly identify and improve deficiencies in the prediction process, thereby significantly improving the accuracy and reliability of SQL generation, optimizing user experience, and providing strong data support for the continuous improvement of language models.

[0064] In some optional embodiments, the accuracy of the predicted structured query language is evaluated according to the semantic analysis result of the predicted structured query language to obtain an evaluation result corresponding to the predicted structured query language, including:

[0065] Obtaining a third execution result corresponding to the predicted structured query language;

[0066] Performing semantic analysis on the predicted structured query language to obtain semantic analysis results;

[0067] The accuracy of the predicted structured query language is evaluated according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language.

[0068] Specifically, in the absence of a target structured query language (SQL) statement as a reference, the solution evaluates its accuracy by executing the predicted SQL and performing in-depth semantic analysis. First, the predicted SQL is executed to obtain the third execution results, which are the actual outputs of the database operation. Next, the predicted SQL is semantically analyzed using natural language processing technology to understand its intent and logical structure and obtain the semantic analysis results. Finally, the accuracy of the predicted SQL is comprehensively evaluated by combining the third execution result and the semantic analysis result. This evaluation process involves comparing whether the execution results are consistent with the expected business logic, and whether the semantics of the SQL correctly express the user's query requirements. The final evaluation results provide quantitative feedback for the accuracy of the predicted SQL.

[0069] In this way, the actual data output is obtained by executing the predicted SQL, and then the semantic analysis technology is used to understand the intent and logic of the statement, ensuring that the generated statement is not only grammatically correct, but also semantically matches the user's query requirements. By comprehensively considering the execution results and semantic analysis results, the accuracy of the predicted SQL can be more accurately evaluated, providing strong support for optimizing the language model and improving the quality of SQL generation.

[0070] In some optional embodiments, the accuracy of the predicted structured query language is evaluated according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language, including:

[0071] Obtain the target semantic features of user needs;

[0072] Matching the semantic features in the semantic analysis result with the target semantic features, and evaluating using the third execution result to obtain a second preliminary evaluation result;

[0073] An evaluation result corresponding to the predicted structured query language is obtained according to the second preliminary evaluation result.

[0074] Specifically, the target semantic features of user needs are first obtained. These features are the key elements of user query intent, such as specific data fields, query conditions, etc. Then, the semantic features extracted from the semantic analysis results of the predicted SQL are matched with the target semantic features of the user needs. This step ensures that the logic of statement generation is consistent with user needs. In addition, combined with the third execution result of the predicted SQL, that is, the output of the actual database query, these matched semantic features are further verified and evaluated to obtain the second preliminary evaluation result. Finally, based on this second preliminary evaluation result, the final evaluation result is obtained. The entire process improves the accuracy of SQL generation by accurately matching semantic features and verifying execution results, ensuring the efficiency of data processing and the precise satisfaction of user needs.

[0075] In this way, by obtaining the target semantic features of user needs, the evaluation process is ensured to be closely connected with the actual needs of users. Then, the semantic analysis results of the predicted SQL are matched with these target semantic features and verified using the third execution result. This step not only verifies the semantic correctness of the statement, but also ensures the consistency of the execution result with user needs. Finally, based on the second preliminary evaluation results, the evaluation results provide comprehensive quantitative feedback on the accuracy of the predicted SQL. This method not only improves the accuracy of SQL generation, but also enhances the intelligence of the system and the trust of users, ensuring the efficiency of data processing and the precise satisfaction of user needs.

[0076] In some optional embodiments, obtaining an evaluation result corresponding to the predicted structured query language according to the second preliminary evaluation result includes:

[0077] Obtaining the number of correct evaluations in the second preliminary evaluation result;

[0078] The ratio between the number of correct evaluations and the total number of second preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

[0079] Specifically, after obtaining the second preliminary evaluation result, since the second preliminary evaluation result includes the results of correct evaluation and the results of incorrect evaluation, the embodiment of the present disclosure needs to count the number of correct evaluations from the second preliminary evaluation result, that is, the number of cases where the predicted SQL fully matches the user's needs in terms of semantic analysis and execution results. Then, the number of correct evaluations is compared with the total number of the second preliminary evaluation results, and a ratio is calculated, which reflects the percentage of correctness of the predicted SQL in all evaluation items. Finally, this ratio result is used as the final evaluation result of the predicted SQL, providing a quantitative measure for the accuracy of the predicted statement. This method makes the evaluation results more objective and accurate, helps to further optimize the language model, improve the accuracy of SQL generation, and ensure that the generated SQL can meet the actual query needs of the user.

[0080] In this way, by carefully analyzing the second preliminary evaluation results, the number of correct evaluations is identified, that is, those prediction statements that match user needs in logic and execution, and then the proportion of these correct evaluations in all evaluation items is calculated, which provides an intuitive indicator for measuring the overall performance of the prediction SQL, thereby providing data support for the optimization and adjustment of the model. Ultimately, this ratio result, as the evaluation result of the prediction SQL, not only enhances the objectivity and accuracy of the evaluation, but also improves the reliability of the automatic SQL generation system, ensuring that user query needs are effectively met.

[0081] In order to understand the technical solution of the present disclosure as a whole, see Figure 2 , Figure 2 It is a flow chart corresponding to the structured query language evaluation method in the embodiment of the present disclosure. The method includes: first receiving the query question input by the user, and generating SQL (also called predicted SQL) according to the query question. Then, it is determined whether the query question has a standard SQL (also called target SQL), so as to decide to adopt one of two different evaluation paths: if there is a standard SQL, the execution results of the standard SQL and the generated SQL in the database are obtained, and the consistency of the SQL and the execution results is compared using a large language model, and then the accuracy of the generated SQL is calculated; if there is no standard SQL, the execution result of the generated SQL in the database is obtained, the large language model is used to perform semantic analysis on the SQL, the large language model is used to evaluate the SQL semantic representation, and finally the accuracy of the generated SQL is calculated.

[0082] It can be seen that the present disclosure proposes a dual evaluation mechanism, which can perform effective SQL accuracy evaluation regardless of whether there is standard SQL, significantly improving the applicability of the method. Secondly, the present disclosure introduces a large language model to compare the consistency of SQL and execution results. This method can more accurately evaluate the accuracy of the generated SQL, and has higher accuracy and robustness than traditional matching methods. Thirdly, the method of the present disclosure can still perform effective SQL accuracy evaluation in complex enterprise environments with limited data or lack of standard SQL, and has a wider range of applicability. This is of great significance for data query and processing in enterprise application scenarios. The method of the present invention can automatically generate SQL and perform evaluation, reduce manual intervention, and improve efficiency and objectivity of evaluation.

[0083] The following describes an apparatus embodiment of the present application, which can be used to execute the structured query language evaluation method in the above embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above embodiment of the structured query language evaluation method of the present application.

[0084] The present disclosure also provides a structured query language evaluation device 300, which is applied to a vehicle terminal, such as Figure 3 As shown, the device comprises:

[0085] The acquisition module 301 is used to acquire the query question input by the user;

[0086] A generating module 302, configured to generate a predictive structured query language corresponding to the query question according to the query question using a large language model;

[0087] The first evaluation module 303 is used to evaluate the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language if there is a target structured query language corresponding to the query question, and obtain an evaluation result corresponding to the predicted structured query language;

[0088] The second evaluation module 304 is used to evaluate the accuracy of the predicted structured query language according to the semantic analysis result of the predicted structured query language if there is no target structured query language corresponding to the query question, and obtain an evaluation result corresponding to the predicted structured query language.

[0089] In some optional embodiments, the first evaluation module 303 evaluates the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language, and obtains an evaluation result corresponding to the predicted structured query language, including:

[0090] Obtaining a first execution result corresponding to the predicted structured query language;

[0091] Acquire a second execution result corresponding to the target structured query language;

[0092] The accuracy of the predicted structured query language is evaluated according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language.

[0093] In some optional embodiments, the first evaluation module 303 evaluates the accuracy of the predicted structured query language according to the first execution result and the second execution result, and obtains an evaluation result corresponding to the predicted structured query language, including:

[0094] Performing a grammatical comparison between the predicted structured query language and the target structured query language to obtain a first comparison result;

[0095] Compare the first execution result and the second execution result for consistency, to obtain a second comparison result;

[0096] Obtaining a first preliminary evaluation result according to the first comparison result and the second comparison result;

[0097] An evaluation result corresponding to the predicted structured query language is obtained according to the first preliminary evaluation result.

[0098] In some optional embodiments, the first evaluation module 303 obtains an evaluation result corresponding to the predicted structured query language according to the first preliminary evaluation result, including:

[0099] Obtain the number of correct evaluations in the first preliminary evaluation result;

[0100] The ratio between the number of correct evaluations and the total number of first preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

[0101] In some optional embodiments, the second evaluation module 304 evaluates the accuracy of the predicted structured query language according to the semantic analysis result of the predicted structured query language, and obtains an evaluation result corresponding to the predicted structured query language, including:

[0102] Obtaining a third execution result corresponding to the predicted structured query language;

[0103] Performing semantic analysis on the predicted structured query language to obtain semantic analysis results;

[0104] The accuracy of the predicted structured query language is evaluated according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language.

[0105] In some optional embodiments, the second evaluation module 304 evaluates the accuracy of the predicted structured query language according to the third execution result and the semantic analysis result, and obtains an evaluation result corresponding to the predicted structured query language, including:

[0106] Obtain the target semantic features of user needs;

[0107] Matching the semantic features in the semantic analysis result with the target semantic features, and evaluating using the third execution result to obtain a second preliminary evaluation result;

[0108] An evaluation result corresponding to the predicted structured query language is obtained according to the second preliminary evaluation result.

[0109] In some optional embodiments, the second evaluation module 304 obtains an evaluation result corresponding to the predicted structured query language according to the second preliminary evaluation result, including:

[0110] Obtaining the number of correct evaluations in the second preliminary evaluation result;

[0111] The ratio between the number of correct evaluations and the total number of second preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

[0112] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0113] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0114] Figure 4A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0115] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0116] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as the structured query language evaluation method. For example, in some embodiments, the structured query language evaluation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the applet distribution described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the structured query language evaluation method by any other appropriate means (e.g., by means of firmware).

[0118] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0120] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0122] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0123] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0124] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0125] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A structured query language evaluation method, the method comprising: Get the query question entered by the user; Using a large language model to generate a predictive structured query language corresponding to the query question according to the query question; If there is a target structured query language corresponding to the query question, then based on the predicted structured query language and the target structured query language, the accuracy of the predicted structured query language is evaluated to obtain an evaluation result corresponding to the predicted structured query language; If there is no target structured query language corresponding to the query question, the accuracy of the predicted structured query language is evaluated according to the semantic analysis result of the predicted structured query language to obtain an evaluation result corresponding to the predicted structured query language.

2. The method according to claim 1, wherein: The step of evaluating the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language to obtain an evaluation result corresponding to the predicted structured query language includes: Acquire a first execution result corresponding to the predictive structured query language; Acquire a second execution result corresponding to the target structured query language; The accuracy of the predicted structured query language is evaluated according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language.

3. The method according to claim 2, wherein: The step of evaluating the accuracy of the predicted structured query language according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language includes: Performing a grammatical comparison between the predicted structured query language and the target structured query language to obtain a first comparison result; Comparing the first execution result and the second execution result for consistency, to obtain a second comparison result; Obtaining a first preliminary evaluation result according to the first comparison result and the second comparison result; An evaluation result corresponding to the predicted structured query language is obtained according to the first preliminary evaluation result.

4. The method according to claim 3, wherein: The obtaining, according to the first preliminary evaluation result, an evaluation result corresponding to the predicted structured query language includes: Obtaining the number of correct evaluations in the first preliminary evaluation results; The ratio between the number of correct evaluations and the total number of the first preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

5. The method according to any one of claims 1 to 4, wherein: The step of evaluating the accuracy of the predicted structured query language according to the semantic analysis result of the predicted structured query language to obtain an evaluation result corresponding to the predicted structured query language includes: Acquire a third execution result corresponding to the predictive structured query language; Performing semantic analysis on the predicted structured query language to obtain a semantic analysis result; The accuracy of the predicted structured query language is evaluated according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language.

6. The method according to claim 5, wherein: The step of evaluating the accuracy of the predicted structured query language according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language includes: Obtain the target semantic features of user needs; Matching the semantic features in the semantic analysis result with the target semantic features, and evaluating using the third execution result to obtain a second preliminary evaluation result; An evaluation result corresponding to the predicted structured query language is obtained according to the second preliminary evaluation result.

7. The method according to claim 6, wherein: The obtaining, according to the second preliminary evaluation result, an evaluation result corresponding to the predicted structured query language includes: Obtaining the number of correct evaluations in the second preliminary evaluation results; The ratio between the number of correct evaluations and the total number of the second preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

8. A structured query language evaluation device, the device comprising: An acquisition module is used to obtain the query question input by the user; A generation module, configured to generate a predictive structured query language corresponding to the query question according to the query question using a large language model; A first evaluation module is used to evaluate the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language if there is a target structured query language corresponding to the query question, so as to obtain an evaluation result corresponding to the predicted structured query language; The second evaluation module is used to evaluate the accuracy of the predicted structured query language according to the semantic analysis result of the predicted structured query language if there is no target structured query language corresponding to the query question, so as to obtain an evaluation result corresponding to the predicted structured query language.

9. The device according to claim 8, wherein: The first evaluation module evaluates the accuracy of the predicted structured query language according to the predicted structured query language and the target structured query language to obtain an evaluation result corresponding to the predicted structured query language, including: Acquire a first execution result corresponding to the predictive structured query language; Acquire a second execution result corresponding to the target structured query language; The accuracy of the predicted structured query language is evaluated according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language.

10. The device according to claim 9, wherein: The first evaluation module evaluates the accuracy of the predicted structured query language according to the first execution result and the second execution result to obtain an evaluation result corresponding to the predicted structured query language, including: Performing a grammatical comparison between the predicted structured query language and the target structured query language to obtain a first comparison result; Comparing the first execution result and the second execution result for consistency, to obtain a second comparison result; Obtaining a first preliminary evaluation result according to the first comparison result and the second comparison result; An evaluation result corresponding to the predicted structured query language is obtained according to the first preliminary evaluation result.

11. The device according to claim 10, wherein: The first evaluation module obtains an evaluation result corresponding to the predicted structured query language according to the first preliminary evaluation result, including: Obtaining the number of correct evaluations in the first preliminary evaluation results; The ratio between the number of correct evaluations and the total number of the first preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

12. The device according to any one of claims 8 to 11, wherein: The second evaluation module evaluates the accuracy of the predicted structured query language according to the semantic analysis result of the predicted structured query language, and obtains an evaluation result corresponding to the predicted structured query language, including: Acquire a third execution result corresponding to the predictive structured query language; Performing semantic analysis on the predicted structured query language to obtain a semantic analysis result; The accuracy of the predicted structured query language is evaluated according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language.

13. The device according to claim 12, wherein: The second evaluation module evaluates the accuracy of the predicted structured query language according to the third execution result and the semantic analysis result to obtain an evaluation result corresponding to the predicted structured query language, including: Obtain the target semantic features of user needs; Matching the semantic features in the semantic analysis result with the target semantic features, and evaluating using the third execution result to obtain a second preliminary evaluation result; An evaluation result corresponding to the predicted structured query language is obtained according to the second preliminary evaluation result.

14. The device according to claim 13, wherein: The second evaluation module obtains an evaluation result corresponding to the predicted structured query language according to the second preliminary evaluation result, including: Obtaining the number of correct evaluations in the second preliminary evaluation results; The ratio between the number of correct evaluations and the total number of the second preliminary evaluation results is calculated, and the ratio result is used as the evaluation result corresponding to the predicted structured query language.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images