Data processing optimization method and device, equipment, storage medium and product

Abstract

The invention discloses a data processing optimization method and device, equipment, a storage medium and a product, relates to the technical field of computers, and discloses an initial structured query language (SQL) script and execution plan information of the initial SQL script; based on the execution plan information, optimizing the initial SQL script through a large language model to obtain at least one candidate SQL script; performing performance testing on the at least one candidate SQL script; and when the processing performance of the tested candidate SQL script is superior to that of the initial SQL script, determining the candidate SQL script as the optimized SQL script. Through the above mode, the optimization efficiency of the SQL script can be improved, the performance optimization period can be shortened, and the optimization quality of the SQL script can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to data processing optimization methods, apparatus, devices, storage media, and products. Background Technology

[0002] SQL (Structured Query Language) is a database language with multiple functions, including data manipulation and data definition. This language is interactive, providing great convenience to users. In computer application systems, a large number of data processing tasks rely on data processing scripts composed of SQL, i.e., SQL scripts. Since the processing performance of SQL scripts largely determines the execution efficiency and quality of data processing tasks, SQL script performance optimization is extremely important.

[0003] In related technologies, SQL script performance optimization requires professional technicians to repeatedly debug based on their professional knowledge and experience, resulting in a long optimization cycle and high labor costs.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a data processing optimization method, apparatus, device, storage medium and product, which aims to solve the technical problems of long performance optimization cycle and high labor cost of SQL scripts.

[0006] To achieve the above objectives, this application proposes a data processing optimization method, the method comprising:

[0007] Obtain the initial Structured Query Language (SQL) script and the execution plan information of the initial SQL script;

[0008] Based on the execution plan information, the initial SQL script is optimized using a large language model to obtain at least one candidate SQL script;

[0009] Perform performance testing on at least one candidate SQL script;

[0010] If the processing performance of a candidate SQL script is better than that of the initial SQL script, the candidate SQL script is determined as the optimized SQL script.

[0011] Optionally, the performance test on the at least one candidate SQL script includes:

[0012] Obtain multiple performance parameters of the initial SQL script and the candidate SQL scripts for testing;

[0013] Based on the acquired multiple performance parameters, the optimization degree parameter of the candidate SQL script relative to the initial SQL script is determined;

[0014] If the optimization parameter meets the reference threshold, the processing performance of the candidate SQL script is determined to be better than that of the initial SQL script.

[0015] Optionally, the multiple performance parameters have various types, and different types of performance parameters are used to describe different processing performance of SQL scripts;

[0016] The step of determining the optimization level parameter of the candidate SQL script relative to the initial SQL script based on the acquired multiple performance parameters includes:

[0017] The performance parameters of the candidate SQL script are compared with the performance parameters of the same type of the initial SQL script to obtain the ratio of the performance parameters.

[0018] The optimization degree parameter is obtained by weighted summation of the ratios of the multiple performance parameters.

[0019] Optionally, based on the execution plan information, the initial SQL script is optimized using a large language model to obtain at least one candidate SQL script, including:

[0020] Based on the execution plan information, the optimization points of the initial SQL script are determined through the large language model;

[0021] Obtain the performance optimization information corresponding to the point to be optimized;

[0022] Based on the performance optimization information, the initial SQL script is optimized using the large language model to obtain at least one candidate SQL script.

[0023] Optionally, determining the optimization points of the initial SQL script based on the execution plan information and the large language model includes:

[0024] Based on the initial SQL script and the execution plan information, a structured first prompt word is generated, which indicates the points to be optimized in the output;

[0025] The first prompt word is input into the large language model to obtain the optimization points of the initial SQL script output by the large language model.

[0026] Optionally, the step of optimizing the initial SQL script based on the performance optimization information and through the large language model to obtain the at least one candidate SQL script includes:

[0027] Based on the initial SQL script, the execution plan information, and the performance optimization information, a structured second prompt word is generated, which indicates the output of candidate SQL scripts;

[0028] The second prompt word is input into the large language model to obtain the at least one candidate SQL script output by the large language model.

[0029] Optionally, obtaining the performance optimization information corresponding to the point to be optimized includes:

[0030] A SQL optimization information database is established, which is used to store performance optimization points of SQL scripts and corresponding performance optimization information;

[0031] Search the SQL optimization information database for the performance optimization information corresponding to the point to be optimized.

[0032] Optionally, based on the execution plan information, the initial SQL script is optimized using a large language model to obtain at least one candidate SQL script, including:

[0033] Obtain negative sample information, which includes candidate SQL scripts generated based on the initial SQL script and execution feedback information of the candidate SQL scripts. The execution feedback information includes data processing logic errors or processing performance inferior to the initial SQL script.

[0034] Based on the execution plan information and the negative sample information, the initial SQL script is optimized using the large language model to obtain at least one candidate SQL script.

[0035] Optionally, the training process of the large language model includes:

[0036] Multiple training samples are obtained, including sample SQL scripts, execution plan information of the sample SQL scripts, and performance optimization information of the sample SQL scripts;

[0037] The large language model is pre-trained using the multiple training samples.

[0038] Optionally, the performance test on the at least one candidate SQL script includes:

[0039] Perform logical tests on the at least one candidate SQL script;

[0040] If the data processing logic of the candidate SQL script is correct, then the candidate SQL script is subjected to performance testing.

[0041] Optionally, the logical test on the at least one candidate SQL script includes:

[0042] The initial SQL script and the candidate SQL script for testing are executed respectively to obtain the data processing results of the initial SQL script and the candidate SQL script;

[0043] If the data processing result of the candidate SQL script is consistent with the data processing result of the initial SQL script, the data processing logic of the candidate SQL script is determined to be correct.

[0044] Optionally, the plurality of performance parameters include:

[0045] The central processing unit (CPU) core usage, memory usage, disk usage, network bandwidth usage, execution time, and lock usage are all measured.

[0046] Furthermore, to achieve the above objectives, this application also proposes a data processing optimization apparatus, which includes:

[0047] The information acquisition module is used to acquire the initial structured query language (SQL) script and the execution plan information of the initial SQL script;

[0048] The script optimization module is used to optimize the initial SQL script based on the execution plan information and through a large language model to obtain at least one candidate SQL script.

[0049] The performance testing module is used to perform performance testing on the at least one candidate SQL script;

[0050] The script determination module is used to determine the candidate SQL script as the optimized SQL script when the processing performance of the tested candidate SQL script is better than that of the initial SQL script.

[0051] Optionally, the performance testing module includes:

[0052] The parameter acquisition unit is used to acquire multiple performance parameters of the initial SQL script and the candidate SQL script being tested;

[0053] A parameter determination unit is used to determine the optimization degree parameter of the candidate SQL script relative to the initial SQL script based on the acquired multiple performance parameters;

[0054] The performance determination unit is used to determine that, if the optimization degree parameter meets the reference threshold, the processing performance of the candidate SQL script is better than that of the initial SQL script.

[0055] Optionally, the multiple performance parameters have various types, and different types of performance parameters are used to describe different processing performance of SQL scripts;

[0056] The parameter determination unit is used to compare multiple performance parameters of the candidate SQL script with multiple performance parameters of the same type of the initial SQL script to obtain multiple performance parameter ratios; and to perform a weighted summation of the multiple performance parameter ratios to obtain the optimization degree parameter.

[0057] Optionally, the script optimization module includes:

[0058] The optimization point determination unit is used to determine the optimization points of the initial SQL script based on the execution plan information and the large language model.

[0059] An information acquisition unit is used to acquire performance optimization information corresponding to the point to be optimized.

[0060] The script optimization unit is used to optimize the initial SQL script based on the performance optimization information and through the large language model to obtain the at least one candidate SQL script.

[0061] Optionally, the optimization point determination unit is used to generate a structured first prompt word based on the initial SQL script and the execution plan information, the first prompt word indicating the output optimization point; and input the first prompt word into the large language model to obtain the optimization point of the initial SQL script output by the large language model.

[0062] Optionally, the script optimization unit is used to generate a structured second prompt word based on the initial SQL script, the execution plan information, and the performance optimization information, wherein the second prompt word indicates the output candidate SQL script; and input the second prompt word into the large language model to obtain the at least one candidate SQL script output by the large language model.

[0063] Optionally, the information acquisition unit is used to determine an SQL optimization information database, which stores performance optimization points of SQL scripts and corresponding performance optimization information; and to search for performance optimization information corresponding to the point to be optimized from the SQL optimization information database.

[0064] Optionally, the script optimization module is used to obtain negative sample information, which includes candidate SQL scripts generated based on the initial SQL script and execution feedback information of the candidate SQL scripts. The execution feedback information includes errors in data processing logic or processing performance inferior to the initial SQL script. Based on the execution plan information and the negative sample information, the initial SQL script is optimized through the large language model to obtain the at least one candidate SQL script.

[0065] Optionally, the training process of the large language model includes:

[0066] Multiple training samples are obtained, including sample SQL scripts, execution plan information of the sample SQL scripts, and performance optimization information of the sample SQL scripts;

[0067] The large language model is pre-trained using the multiple training samples.

[0068] Optionally, the performance testing module includes:

[0069] A logic testing unit is used to perform logical tests on the at least one candidate SQL script;

[0070] The performance testing unit is used to perform performance testing on the candidate SQL scripts when the data processing logic of the tested candidate SQL scripts is correct.

[0071] Optionally, the logic testing unit is configured to execute the initial SQL script and the candidate SQL script for testing respectively, and obtain the data processing results of the initial SQL script and the candidate SQL script; if the data processing result of the candidate SQL script is consistent with the data processing result of the initial SQL script, the data processing logic of the candidate SQL script is determined to be correct.

[0072] Optionally, the plurality of performance parameters include:

[0073] The central processing unit (CPU) core usage, memory usage, disk usage, network bandwidth usage, execution time, and lock usage are all measured.

[0074] In addition, to achieve the above objectives, this application also proposes a data processing optimization apparatus, the apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data processing optimization method as described above.

[0075] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the data processing optimization method described above.

[0076] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the data processing optimization method described above.

[0077] One or more technical solutions proposed in this application have at least the following technical effects:

[0078] The optimization scheme provided in this application obtains the initial SQL script and its execution plan information. Based on the execution plan information, the initial SQL script can be optimized using a large language model to obtain at least one candidate SQL script. This method of obtaining candidate SQL scripts eliminates the need for repeated manual debugging, saving manpower costs and improving efficiency. Performance testing is then performed on the at least one candidate SQL script. If the processing performance of a candidate SQL script is superior to that of the initial SQL script, it is identified as the optimized SQL script. This ensures that the candidate SQL script outperforms the initial SQL script, thus guaranteeing both optimization efficiency and quality while shortening the performance optimization cycle. Attached Figure Description

[0079] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0080] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0081] Figure 1 This is a schematic diagram of an implementation environment for the data processing optimization method of this application;

[0082] Figure 2 This is a flowchart illustrating the first embodiment of the data processing optimization method of this application.

[0083] Figure 3This is a flowchart illustrating the second embodiment of the data processing optimization method of this application.

[0084] Figure 4 This is a flowchart illustrating the third embodiment of the data processing optimization method of this application.

[0085] Figure 5 This is a schematic diagram of an SQL script optimization process provided in this application;

[0086] Figure 6 This is a schematic diagram illustrating the process of generating candidate SQL scripts using a model provided in this application;

[0087] Figure 7 This is a schematic diagram of the module structure of the data processing optimization device according to an embodiment of this application;

[0088] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the data processing optimization method in the embodiments of this application.

[0089] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0090] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0091] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0092] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application. See also... Figure 1 The implementation environment includes a terminal 101 and a server 102. The terminal 101 and server 102 are connected via a wireless or wired network. For example, the terminal 101 may be a computer, mobile phone, tablet computer, or other terminal. For example, the server 102 may be a backend server for an application or a cloud server providing services such as cloud computing and cloud storage.

[0093] In this embodiment, terminal 101 is used to obtain an initial SQL script and its execution plan information, and send the initial SQL script and its execution plan information to server 102. Server 102 is used to receive the SQL script and its execution plan information, optimize the initial SQL script based on the execution plan information using a large language model, obtain at least one candidate SQL script, and send the at least one candidate SQL script to terminal 101. Terminal 101 is used to perform performance testing on the at least one candidate SQL script, and if the processing performance of the tested candidate SQL script is better than that of the initial SQL script, the candidate SQL script is determined as the optimized SQL script.

[0094] Alternatively, server 102, after obtaining at least one candidate SQL script, performs performance testing on the candidate SQL script. If the processing performance of a candidate SQL script is superior to that of the initial SQL script, the candidate SQL script is identified as the optimized SQL script. The optimized SQL script is then sent to terminal 101. Terminal 101 receives and stores the optimized SQL script.

[0095] Alternatively, after obtaining the initial SQL script and its execution plan information, terminal 101 optimizes the initial SQL script locally based on the execution plan information using a large language model to obtain at least one candidate SQL script; performs performance testing on the at least one candidate SQL script; and determines the candidate SQL script as the optimized SQL script when the processing performance of the tested candidate SQL script is better than that of the initial SQL script.

[0096] Figure 2 This is a flowchart illustrating the first embodiment of the data processing optimization method of this application. (Refer to...) Figure 2 Taking the terminal as the executing entity as an example, this data processing optimization method includes the following steps S10 to S40:

[0097] Step S10: Obtain the initial Structured Query Language (SQL) script and the execution plan information of the initial SQL script.

[0098] SQL is a standardized programming language used for managing and manipulating databases. An SQL script is a data processing script composed of SQL statements. For example, the initial SQL script can be obtained by either retrieving a user-inputted SQL script or retrieving it from a database. The SQL scripts stored in the database are written by technical personnel.

[0099] The execution plan information includes the execution steps of the initial SQL script and the execution time of each step. This execution plan information is obtained by querying an interpreter tool within the SQL execution environment to retrieve the execution plan information related to the initial SQL. For example, the execution plan information includes the table access order, join type, index usage, scan method, and the actual time and number of rows for each step. For example, interpreters include EXPLAIN (an interpreter tool) for MySQL (a relational database management system) and EXPLAIN ANALYZE (an interpreter tool) for PostgreSQL (a relational database management system).

[0100] Step S20: Based on the execution plan information, the initial SQL script is optimized using a large language model to obtain at least one candidate SQL script.

[0101] Large Language Models (LLMs) are trained models that possess knowledge of SQL script writing and optimization. They can optimize an initial SQL script based on the execution plan information to obtain at least one candidate SQL script.

[0102] Optionally, the large language model includes GPT-4 (Generative Pre-trained Transformer 4), a large-scale multimodal model that can accept image and text input and produce text output. Of course, the large language model can also be other specially optimized models, and this application embodiment does not limit this.

[0103] Optionally, the training process of the large language model includes: acquiring multiple training samples, which include sample SQL scripts, execution plan information of sample SQL scripts, and performance optimization information of sample SQL scripts; and pre-training the large language model using multiple training samples.

[0104] For example, in addition to the sample SQL script, its execution plan information, and performance optimization information, the training samples may also include other information, such as the optimization points of the sample SQL script, the data processing results of the sample SQL script, the references used to write the sample SQL script, and the optimized SQL script corresponding to the sample SQL script. Furthermore, this application does not limit the specific implementation method for pre-training the large language model.

[0105] In this embodiment of the application, since the training samples include sample SQL scripts, execution plan information of sample SQL scripts, and performance optimization information of sample SQL scripts, after training the large language model with multiple training samples, the model can learn performance optimization knowledge such as the SQL scripts and the correlation information between the SQL scripts' execution plan information and performance optimization information. Subsequently, based on the SQL scripts and their execution plan information, the initial SQL scripts can be optimized using the learned performance optimization knowledge to output candidate SQL scripts.

[0106] Step S30: Perform performance testing on at least one candidate SQL script.

[0107] Since SQL script performance involves multiple aspects, performance testing of at least one candidate SQL script includes performing multiple types of performance tests on it. These different types of tests are used to determine the different processing performance characteristics of the SQL script. For example, testing the SQL script's performance in terms of CPU (Central Processing Unit) core usage, memory usage, disk space usage, network bandwidth usage, execution time, and lock usage, among other aspects.

[0108] Step S40: If the processing performance of the candidate SQL script is better than that of the initial SQL script, the candidate SQL script is determined as the optimized SQL script.

[0109] Among them, the processing performance of the candidate SQL script is better than that of the initial SQL script, including: one or more candidate SQL scripts have better processing performance than the initial SQL script.

[0110] Optionally, after the terminal determines the candidate SQL script as the optimized SQL script, it upgrades and updates the SQL script in the database, that is, it updates the initial SQL script stored in the database to the optimized SQL script.

[0111] The optimization scheme provided in this application obtains the initial SQL script and its execution plan information. Based on the execution plan information, the initial SQL script can be optimized using a large language model to obtain at least one candidate SQL script. This method of obtaining candidate SQL scripts eliminates the need for repeated manual debugging, saving manpower costs and improving efficiency. Performance testing is then performed on the at least one candidate SQL script. If the processing performance of a candidate SQL script is superior to that of the initial SQL script, it is identified as the optimized SQL script. This ensures that the candidate SQL script outperforms the initial SQL script, thus guaranteeing both optimization efficiency and quality while shortening the performance optimization cycle.

[0112] Based on the first embodiment described above, a second embodiment of this application is proposed. Contents that are the same as or similar to the first embodiment can be referred to the above description and will not be repeated hereafter. (Refer to...) Figure 3 In the second embodiment, step S20 includes steps S201 to S203:

[0113] Step S201: Based on the execution plan information, determine the optimization points of the initial SQL script through the large language model.

[0114] The areas to be optimized are the parts of the initial SQL script that need improvement; they can also be understood as the parts that limit the processing performance of the initial SQL script. By optimizing these areas, the processing performance of the SQL script can be improved.

[0115] Optionally, based on the execution plan information, the optimization points of the initial SQL script are determined through a large language model, including: generating a structured first prompt word based on the initial SQL script and execution plan information, the first prompt word indicating the optimization points to be output; inputting the first prompt word into the large language model to obtain the optimization points of the initial SQL script output by the large language model.

[0116] Structured prompts are input text used when using large language models to guide the model in generating specific types of output. Structured prompts typically contain explicit instructions and contextual information, making the generated text more aligned with user expectations. In this application, the first prompt indicates the points to be optimized in the initial SQL script based on the execution plan information. Furthermore, this application does not impose any restrictions on the specific content and structure of the first prompt.

[0117] In this embodiment, since the large language model has knowledge of SQL script writing and optimization, it generates a structured first prompt word based on the initial SQL script and execution plan information. The first prompt word is then input into the large language model, which can identify the optimization points of the initial SQL script based on the first prompt word. This ensures the accuracy of the optimization points output by the large language model and improves the efficiency of determining the optimization points of the initial SQL script.

[0118] It should be noted that in other embodiments, it is not necessary to use a large language model to determine the optimization points of the initial SQL script. For example, the optimization points of the initial SQL script can be determined by technical personnel analyzing the execution plan information.

[0119] Step S202: Obtain the performance optimization information corresponding to the point to be optimized.

[0120] Performance optimization information, also known as performance optimization strategy, includes the methods or specific steps to optimize the points to be optimized.

[0121] Optionally, obtaining performance optimization information corresponding to the point to be optimized includes: determining an SQL optimization information database, which stores performance optimization points of SQL scripts and their corresponding performance optimization information; and searching for the performance optimization information corresponding to the point to be optimized from the SQL optimization information database. For example, knowledge retrieval technology can be used to retrieve the performance optimization information corresponding to the point to be optimized from the SQL optimization information database through keyword matching or semantic search.

[0122] The SQL optimization information database stores a large amount of performance optimization information and corresponding optimization points. This means that once the optimization points of the initial SQL script are determined, the corresponding performance optimization information can be accurately determined based on the SQL optimization information database, greatly improving the efficiency of determining performance optimization information.

[0123] Step S203: Based on performance optimization information, optimize the initial SQL script using a large language model to obtain at least one candidate SQL script.

[0124] Optionally, this step is implemented as follows: based on the initial SQL script, execution plan information, and performance optimization information, a structured second prompt word is generated, the second prompt word indicating the output candidate SQL script; the second prompt word is input into the large language model to obtain at least one candidate SQL script output by the large language model.

[0125] The second prompt word in this application indicates that at least one candidate SQL script corresponding to the initial SQL script should be output based on the execution plan information and performance optimization information of the initial SQL script. In addition, this application does not limit the specific content and structure of the second prompt word.

[0126] In this embodiment, since the large language model possesses knowledge in the field of SQL script writing and optimization, it generates structured second prompt words based on the initial SQL script, its execution plan information, and performance optimization information. These second prompt words are then input into the large language model, allowing it to optimize the initial SQL script and obtain at least one candidate SQL script. This ensures both the quality of the candidate SQL scripts output by the large language model and improves the efficiency of obtaining candidate SQL scripts.

[0127] In this embodiment of the application, based on the execution plan information, the optimization points of the initial SQL script are determined through the large language model, and the performance optimization information corresponding to the optimization points is obtained. In this way, the model can optimize the initial SQL script based on the performance optimization information and obtain at least one candidate SQL script, thereby improving the efficiency of obtaining candidate SQL scripts.

[0128] It should be noted that, in addition to the execution plan information and performance optimization information used in the above scheme, other information can also be used to assist the large language model in generating candidate SQL scripts. For example, based on the above scheme, negative sample information can be further added to assist the large language model in generating candidate SQL scripts. Accordingly, based on the execution plan information, the initial SQL script is optimized through the large language model to obtain at least one candidate SQL script, including the following steps (A) and (B):

[0129] Step (A): Obtain negative sample information, which includes candidate SQL scripts generated based on the initial SQL script and execution feedback information of the candidate SQL scripts.

[0130] Among them, the execution feedback information in the negative sample information includes errors in data processing logic or processing performance that is inferior to the initial SQL script.

[0131] One point to note is that when generating candidate SQL scripts based on a large language model, only one candidate SQL script can be generated at a time, and then the execution feedback information of that candidate SQL script can be obtained. When generating subsequent candidate SQL scripts, the execution feedback information of the first candidate SQL script can be incorporated.

[0132] For example, obtaining execution feedback information of candidate SQL scripts includes: running candidate SQL scripts and initial SQL scripts; obtaining performance parameters of candidate SQL scripts and initial SQL scripts during the running process; obtaining data processing results of candidate SQL scripts and initial SQL scripts after the running is completed; and determining execution feedback information of candidate SQL scripts based on the obtained performance parameters and data processing results.

[0133] For example, if the data processing results of the candidate SQL script are inconsistent with those of the initial SQL script, the execution feedback information of the candidate SQL script is determined to be due to an error in the data processing logic. If the data processing results of the candidate SQL script are consistent with those of the initial SQL script, but the performance parameters of the candidate SQL script are lower than those of the initial SQL script, the execution feedback information of the candidate SQL script is determined to be due to inferior processing performance compared to the initial SQL script. Then, the candidate SQL script and its execution feedback information are used as negative sample information to generate candidate SQL scripts for the next round. Of course, if the data processing results of the candidate SQL script are consistent with those of the initial SQL script, and the processing performance of the candidate SQL script is superior to that of the initial SQL script, then the candidate SQL script can be identified as the optimized SQL script.

[0134] Step (B): Based on the execution plan information and negative sample information, optimize the initial SQL script using a large language model to obtain at least one candidate SQL script.

[0135] Optionally, this step is implemented as follows: based on the initial SQL script, execution plan information, performance optimization information, and negative sample information, a structured second prompt word is generated, which indicates the output candidate SQL script; the second prompt word is input into the large language model to obtain at least one candidate SQL script output by the large language model.

[0136] In this embodiment, for candidate SQL scripts generated by the large language model based on the initial SQL script, if their data processing logic is flawed or their processing performance is inferior to the initial SQL script, they are not directly discarded. Instead, the candidate SQL script and its execution feedback information are used to form negative sample information, which is then input into the large language model along with the execution plan information to assist the model in generating subsequent candidate SQL scripts. By increasing the dimensionality of the information input to the large language model, the quality of the candidate SQL scripts generated by the large language model is further improved.

[0137] Based on the first embodiment of this application described above, a third embodiment of this application is proposed. (Refer to...) Figure 4 In the third embodiment, step S30 includes steps S301 to S303:

[0138] Step S301: Obtain multiple performance parameters of the initial SQL script and the candidate SQL scripts for testing.

[0139] For example, several performance parameters include: number of CPU cores used, memory usage, disk usage, network bandwidth usage, execution time, and lock usage.

[0140] The number of CPU cores used includes the number of CPU cores consumed by SQL scripts for statement parsing and complex calculations during queries. For example, this includes the number of CPU cores consumed by sorting, aggregation, and calculating column values.

[0141] Memory space usage includes: the memory space used by the database engine to cache recently executed query plans during SQL script execution; and the memory space used to store temporary data during operations such as sorting and aggregation, such as the memory space used to store temporary data in hash tables and sorting buffers.

[0142] Disk space usage includes: the amount of disk space used for inserting, updating or deleting data in the table, synchronizing and updating related index files, and uploading related data to the disk.

[0143] Network bandwidth usage includes the network bandwidth usage of data transmission across nodes in a distributed database.

[0144] Execution time refers to the time consumed from the start to the end of the execution of an SQL script.

[0145] Lock usage includes the amount of various locks used during SQL script execution, such as row locks, page locks, table locks, shared locks, exclusive locks, optimistic locks, and pessimistic locks. In high-concurrency environments, multiple queries may compete for the same resources such as tables, rows, and indexes, leading to lock waiting. This not only affects query response time but may also cause resource idleness. Therefore, setting lock usage as a performance parameter to measure the quality of script optimization can result in SQL scripts with more comprehensive performance optimization.

[0146] In this embodiment, multiple types of performance parameters, including CPU core usage, memory space usage, disk space usage, network bandwidth usage, execution time, and lock usage, are obtained, which can ensure that the optimized SQL script obtained based on the performance parameters is improved in multiple performance aspects.

[0147] Step S302: Based on the acquired multiple performance parameters, determine the optimization parameters of the candidate SQL script relative to the initial SQL script.

[0148] The optimization level parameter indicates the degree of optimization of the candidate SQL script relative to the initial SQL script.

[0149] Multiple performance parameters come in various types, each used to describe different processing performance parameters of an SQL script. For example, execution time describes the execution efficiency of an SQL script, network bandwidth usage describes the network usage of an SQL script, and memory usage describes the memory usage of an SQL script.

[0150] Optionally, based on the acquired multiple performance parameters, the optimization degree parameter of the candidate SQL script relative to the initial SQL script is determined, including: comparing the multiple performance parameters of the candidate SQL script with the multiple performance parameters of the same type of the initial SQL script to obtain multiple performance parameter ratios; and performing a weighted summation of the multiple performance parameter ratios to obtain the optimization degree parameter.

[0151] For example, the optimization parameter can be determined using the following relationship:

[0152]

[0153] Where performance_Score is the optimization level parameter. The ratio of CPU core usage between the candidate SQL script and the initial SQL script. The ratio of the memory usage of the candidate SQL script to that of the initial SQL script. The ratio of disk space usage of the candidate SQL script to that of the initial SQL script. The ratio of network bandwidth usage between the candidate SQL script and the initial SQL script. The ratio of the execution time of the candidate SQL script to that of the initial SQL script. This represents the ratio of lock usage between the candidate SQL script and the initial SQL script. w1, w2, w3, w4, w5, and w6 are weights, and the values ​​of each weight can be the same or different; this embodiment does not impose any restrictions on this.

[0154] Step S303: If the optimization degree parameter meets the reference threshold, determine that the processing performance of the candidate SQL script is better than that of the initial SQL script.

[0155] For example, the reference threshold is 1, and the optimization degree parameter is less than 1, meaning the optimization degree parameter meets the reference threshold. It can be understood that when the optimization degree parameter is less than this reference threshold, it indicates that the processing performance of the candidate SQL script is better than the initial SQL script, and the closer the optimization degree parameter is to 0, the greater the optimization degree of the candidate SQL script relative to the initial SQL script.

[0156] In this embodiment of the application, multiple performance parameters of the candidate SQL script are compared with multiple performance parameters of the same type of the initial SQL script to obtain multiple performance parameter ratios; the multiple performance parameter ratios are weighted and summed to obtain the optimization degree parameter. The optimization degree parameter can accurately measure the performance optimization degree of the candidate SQL script relative to the initial SQL script, so as to select the candidate SQL script with a higher performance optimization degree as the optimized SQL script.

[0157] In this embodiment of the application, by using multiple performance parameters of the initial SQL script and the candidate SQL script for testing to construct optimization degree parameters, the optimization degree of the candidate SQL script relative to the initial SQL script is quantified, and a reference threshold is used as the judgment standard, which makes it easier to more accurately determine the performance optimization of the candidate SQL script relative to the initial SQL script.

[0158] It should be noted that, in addition to the methods described above, optimization parameters can be determined in other ways based on multiple performance parameters, and this application does not limit such methods.

[0159] It should be noted that the above describes the specific process of performance testing. Optionally, in some embodiments, before conducting performance testing, at least one candidate SQL script is first subjected to logical testing. If the data processing logic of the candidate SQL script is correct, the candidate SQL script is then subjected to performance testing. This ensures that the optimized SQL script determined based on the test results is not only superior to the initial SQL script in terms of processing performance, but also that the data processing logic is correct.

[0160] Optionally, at least one candidate SQL script is subjected to logical testing, including: executing the initial SQL script and the candidate SQL script to be tested respectively, and obtaining the data processing results of the initial SQL script and the candidate SQL script; if the data processing result of the candidate SQL script is consistent with the data processing result of the initial SQL script, the data processing logic of the candidate SQL script is determined to be correct.

[0161] In this embodiment, the initial SQL script is written by a technician, and its data processing result has been verified as correct. If the data processing result of the candidate SQL script is consistent with that of the initial SQL script, it indicates that the data processing logic of the candidate SQL script is correct. Therefore, by obtaining and comparing the data processing results of the candidate SQL script and the initial SQL script, it is possible to accurately determine whether the data processing logic of the candidate SQL script is correct.

[0162] Figure 5 This is a schematic diagram illustrating an SQL script optimization process provided in this application. (Reference) Figure 5 First, an initial SQL script and candidate SQL scripts generated using a large language model are obtained. Then, both the initial and candidate SQL scripts are submitted to the SQL execution environment and run. The data processing results of the initial and candidate SQL scripts are collected and stored. The correctness of the candidate SQL script's data processing results is determined. If the results are inconsistent with the initial SQL script, the candidate SQL script's processing logic is incorrect, and a new round of candidate SQL script generation is initiated. If the results are consistent with the initial SQL script, the candidate SQL script's processing logic is correct. Performance measurement of the candidate SQL script is performed using performance parameters collected during script execution to determine its optimization level. Based on these optimization parameters, it is determined whether the candidate SQL script performs better than the initial SQL script. If the candidate SQL script performs better, it is saved as the optimized SQL script, and the initial SQL script is then updated to reflect this optimized performance.

[0163] It should be noted that during the process of running the initial SQL script and candidate SQL scripts, if the process has not yet ended, that is, before the data processing results are available, and it can be determined based on the collected performance parameters that the processing performance of the candidate SQL script is inferior to that of the initial SQL script, then the running of the candidate SQL script will be stopped, and the next round of candidate SQL script generation will begin, in order to avoid invalid SQL scripts consuming resources.

[0164] Another point to note is that in some embodiments, the SQL script optimization process is completed after updating the initial SQL script. In other embodiments, the above process continues based on the updated SQL script; that is, a new round of script optimization is performed until the optimized SQL script meets the requirements.

[0165] Figure 6 This is a schematic diagram illustrating the process of generating candidate SQL scripts using a model provided in this application. (Reference) Figure 6First, the initial SQL script is obtained. Then, the execution plan information of the initial SQL script is retrieved from the SQL execution environment. Next, using a large language model, the optimization points of the initial SQL script are identified based on the execution plan information. Then, the performance optimization information corresponding to the optimization points is retrieved from the SQL optimization information database, and the previous version of the candidate SQL script and its execution feedback information are obtained. Based on the obtained initial SQL script, execution plan information, performance optimization information, previous version of the candidate SQL script and its execution feedback information, model prompt words are constructed. The prompt words are input into the large language model to instruct the model to output candidate SQL scripts.

[0166] It's important to note that the processing and statistical analysis of business data provides data support for intelligent business operations and scientific decision-making. Many data processing tasks rely on scripts written by data engineers, such as SQL scripts. Assuming the processing logic is correct, improving the performance of SQL scripts is crucial and valuable. This helps save processing time to meet the timeliness requirements of business data and reduces resource consumption. However, SQL scripts developed by data engineers, even with correct processing logic, often lack professional performance optimization, resulting in low efficiency. This increases hardware resource consumption and fails to meet the timeliness requirements of the business. Optimizing SQL script performance based on the expertise and tuning experience of technical personnel heavily relies on specialized technical staff, making it difficult to popularize and implement. Furthermore, repeated manual debugging increases development time and costs.

[0167] This application leverages a large language model to construct a pipeline for SQL script performance optimization, realizing an iterative and automated SQL script optimization process. This makes SQL script performance optimization more efficient and intelligent, reduces reliance on specialized technical personnel, and democratizes SQL script performance optimization, facilitating wider-scale SQL performance optimization. Furthermore, the optimized SQL scripts consume fewer resources, reducing hardware costs, achieving higher operating efficiency, and better meeting business requirements for the timeliness of data processing results.

[0168] Another point to note is that the above examples are only for understanding this application and do not constitute a limitation on the data processing optimization method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0169] This application also provides a data processing optimization device, please refer to... Figure 7 The data processing optimization device includes:

[0170] The information acquisition module 10 is used to acquire the initial structured query language SQL script and the execution plan information of the initial SQL script;

[0171] The script optimization module 20 is used to optimize the initial SQL script based on the execution plan information and through a large language model to obtain at least one candidate SQL script.

[0172] Performance testing module 30 is used to perform performance testing on at least one candidate SQL script;

[0173] The script determination module 40 is used to determine the candidate SQL script as the optimized SQL script when the processing performance of the tested candidate SQL script is better than that of the initial SQL script.

[0174] Optionally, the performance testing module 30 includes:

[0175] The parameter acquisition unit is used to acquire multiple performance parameters of the initial SQL script and the candidate SQL scripts for testing;

[0176] The parameter determination unit is used to determine the degree of optimization of the candidate SQL script relative to the initial SQL script based on multiple acquired performance parameters.

[0177] The performance determination unit is used to determine whether the processing performance of the candidate SQL script is better than that of the initial SQL script, provided that the optimization degree parameter meets the reference threshold.

[0178] Optionally, multiple performance parameters have various types, and different types of performance parameters are used to describe different processing performance of SQL scripts;

[0179] The parameter determination unit is used to compare multiple performance parameters of the candidate SQL script with multiple performance parameters of the same type of the initial SQL script to obtain multiple performance parameter ratios; and to perform a weighted summation of the multiple performance parameter ratios to obtain the optimization degree parameter.

[0180] Optionally, the script optimization module 20 includes:

[0181] The optimization point determination unit is used to determine the optimization points of the initial SQL script based on the execution plan information and through the large language model;

[0182] The information acquisition unit is used to acquire performance optimization information corresponding to the point to be optimized.

[0183] The script optimization unit is used to optimize the initial SQL script based on performance optimization information and through a large language model to obtain at least one candidate SQL script.

[0184] Optionally, the optimization point determination unit is used to generate a structured first prompt word based on the initial SQL script and execution plan information. The first prompt word indicates the output optimization point. The first prompt word is input into the large language model to obtain the optimization point of the initial SQL script output by the large language model.

[0185] Optionally, the script optimization unit is used to generate a structured second prompt word based on the initial SQL script, execution plan information, and performance optimization information. The second prompt word indicates the output candidate SQL script. The second prompt word is input into the large language model to obtain at least one candidate SQL script output by the large language model.

[0186] Optionally, the information acquisition unit is used to determine the SQL optimization information database, which stores the performance optimization points of SQL scripts and the corresponding performance optimization information; and to search for the performance optimization information corresponding to the point to be optimized from the SQL optimization information database.

[0187] Optionally, the script optimization module 20 is used to obtain negative sample information, which includes candidate SQL scripts generated based on the initial SQL script and execution feedback information of the candidate SQL scripts. The execution feedback information includes errors in data processing logic or processing performance inferior to the initial SQL script. Based on the execution plan information and negative sample information, the initial SQL script is optimized through a large language model to obtain at least one candidate SQL script.

[0188] Optionally, the training process of a large language model includes:

[0189] Obtain multiple training samples, including sample SQL scripts, execution plan information of sample SQL scripts, and performance optimization information of sample SQL scripts;

[0190] The large language model is pre-trained using multiple training samples.

[0191] Optionally, the performance testing module 30 includes:

[0192] A logic test unit is used to perform logical tests on at least one candidate SQL script.

[0193] The performance testing unit is used to perform performance tests on candidate SQL scripts when the data processing logic of the candidate SQL scripts is correct.

[0194] Optionally, the logic test unit is used to execute the initial SQL script and the candidate SQL script for testing respectively, and obtain the data processing results of the initial SQL script and the candidate SQL script; if the data processing result of the candidate SQL script is consistent with the data processing result of the initial SQL script, the data processing logic of the candidate SQL script is determined to be correct.

[0195] Optionally, several performance parameters include:

[0196] The central processing unit (CPU) core usage, memory usage, disk usage, network bandwidth usage, execution time, and lock usage are all measured.

[0197] The optimization scheme provided in this application obtains the initial SQL script and its execution plan information. Based on the execution plan information, the initial SQL script can be optimized using a large language model to obtain at least one candidate SQL script. This method of obtaining candidate SQL scripts eliminates the need for repeated manual debugging, saving manpower costs and improving efficiency. Performance testing is then performed on the at least one candidate SQL script. If the processing performance of a candidate SQL script is superior to that of the initial SQL script, it is identified as the optimized SQL script. This ensures that the candidate SQL script outperforms the initial SQL script, thus guaranteeing both optimization efficiency and quality while shortening the performance optimization cycle.

[0198] The data processing optimization apparatus provided in this application, employing the data processing optimization method described in the above embodiments, can solve the technical problems of long performance optimization cycles and high labor costs associated with SQL scripts. Compared with the prior art, the beneficial effects of the data processing optimization apparatus provided in this application are the same as those of the data processing optimization method described in the above embodiments, and other technical features in the data processing optimization apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0199] This application provides a data processing optimization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data processing optimization method in the first embodiment described above.

[0200] The following is for reference. Figure 8This document illustrates a structural schematic diagram of a data processing optimization device suitable for implementing embodiments of this application. The data processing optimization device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The data processing optimization device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0201] like Figure 8 As shown, the data processing optimization device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the data processing optimization device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the data processing optimization device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a data processing optimization device with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0202] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0203] The data processing optimization device provided in this application, employing the data processing optimization method described in the above embodiments, can solve the technical problems of long performance optimization cycles and high labor costs associated with SQL scripts. Compared with the prior art, the beneficial effects of the data processing optimization device provided in this application are the same as those of the data processing optimization method described in the above embodiments, and other technical features of this data processing optimization device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0204] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0205] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0206] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the data processing optimization method described in the above embodiments.

[0207] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0208] The aforementioned computer-readable storage medium may be included in the data processing optimization device; or it may exist independently and not assembled into the data processing optimization device.

[0209] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the data processing optimization device, the data processing optimization device: obtains an initial Structured Query Language (SQL) script and its execution plan information; optimizes the initial SQL script based on the execution plan information using a large language model to obtain at least one candidate SQL script; performs performance testing on the at least one candidate SQL script; and determines the candidate SQL script as the optimized SQL script when the processing performance of the tested candidate SQL script is better than that of the initial SQL script.

[0210] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0211] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0212] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0213] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described data processing optimization method. This solves the technical problems of long performance optimization cycles and high labor costs associated with SQL scripts. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the data processing optimization method provided in the above embodiments, and will not be elaborated upon here.

[0214] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data processing optimization method described above.

[0215] The computer program product provided in this application can solve the technical problems of long performance optimization cycles and high labor costs for SQL scripts. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the data processing optimization method provided in the above embodiments, and will not be repeated here.

[0216] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A data processing optimization method, characterized in that, The method includes: Obtain the initial Structured Query Language (SQL) script and the execution plan information of the initial SQL script; Based on the execution plan information, the initial SQL script is optimized using a large language model to obtain at least one candidate SQL script; Perform performance testing on at least one candidate SQL script; If the processing performance of a candidate SQL script is better than that of the initial SQL script, the candidate SQL script is determined as the optimized SQL script.

2. The method as described in claim 1, characterized in that, The performance test of the at least one candidate SQL script includes: Obtain multiple performance parameters of the initial SQL script and the candidate SQL scripts for testing; Based on the acquired multiple performance parameters, the optimization degree parameter of the candidate SQL script relative to the initial SQL script is determined; If the optimization parameter meets the reference threshold, the processing performance of the candidate SQL script is determined to be better than that of the initial SQL script.

3. The method as described in claim 2, characterized in that, The multiple performance parameters have various types, and different types of performance parameters are used to describe the different processing performance of SQL scripts; The step of determining the optimization level parameter of the candidate SQL script relative to the initial SQL script based on the acquired multiple performance parameters includes: The performance parameters of the candidate SQL script are compared with the performance parameters of the same type of the initial SQL script to obtain the ratio of the performance parameters. The optimization degree parameter is obtained by weighted summation of the ratios of the multiple performance parameters.

4. The method as described in claim 1, characterized in that, Based on the execution plan information, the initial SQL script is optimized using a large language model to obtain at least one candidate SQL script, including: Based on the execution plan information, the optimization points of the initial SQL script are determined through the large language model; Obtain the performance optimization information corresponding to the point to be optimized; Based on the performance optimization information, the initial SQL script is optimized using the large language model to obtain at least one candidate SQL script.

5. The method as described in claim 4, characterized in that, The step of determining the optimization points of the initial SQL script based on the execution plan information and the large language model includes: Based on the initial SQL script and the execution plan information, a structured first prompt word is generated, which indicates the points to be optimized in the output; The first prompt word is input into the large language model to obtain the optimization points of the initial SQL script output by the large language model.

6. The method as described in claim 4, characterized in that, The process of optimizing the initial SQL script based on the performance optimization information and using the large language model to obtain the at least one candidate SQL script includes: Based on the initial SQL script, the execution plan information, and the performance optimization information, a structured second prompt word is generated, which indicates the output of candidate SQL scripts; The second prompt word is input into the large language model to obtain the at least one candidate SQL script output by the large language model.

7. A data processing optimization device, characterized in that, The device includes: The information acquisition module is used to acquire the initial structured query language (SQL) script and the execution plan information of the initial SQL script; The script optimization module is used to optimize the initial SQL script based on the execution plan information and through a large language model to obtain at least one candidate SQL script. The performance testing module is used to perform performance testing on the at least one candidate SQL script; The script determination module is used to determine the candidate SQL script as the optimized SQL script when the processing performance of the tested candidate SQL script is better than that of the initial SQL script.

8. A data processing optimization device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data processing optimization method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the data processing optimization method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the data processing optimization method as described in any one of claims 1 to 6.

Images