SQL (Structured Query Language) retrieval method based on small parameter model

Abstract

The invention relates to the technical field of SQL (Structured Query Language) databases, in particular to an SQL retrieval method based on a small parameter model, which comprises the following steps: carrying out model optimization and parameter adjustment to form a small model in the vertical class field, and forming a knowledge graph for a to-be-retrieved database; respectively inputting cue words into the vertical domain small model and the knowledge graph to obtain a first sql statement and a second sql statement; and merging to obtain a third sql statement for retrieval. In order to solve the problem of poor effect of deploying small models under the condition of low computing power in the prior art, for a first model with fewer parameters, firstly, model optimization and parameter adjustment are performed on a to-be-retrieved database to form a small model in the vertical class field, and a knowledge graph corresponding to the to-be-retrieved database is configured to form auxiliary information. In the subsequent processing process, the sql statements and the corresponding auxiliary information can be obtained by questioning the vertical domain small model and the knowledge graph, and then synthesis is performed through the second model, so that accurate generation of the sql statements under the low-parameter model is realized.

Description

Technical Field

[0001] This invention relates to the field of SQL database technology, and specifically to an SQL retrieval method based on a small parameter model. Background Technology

[0002] Structured Query Language (SQL) is a special-purpose programming language used for database querying and programming. It's used to access, query, update, and manage relational database systems. SQL is a high-level, non-procedural programming language that allows users to work on high-level data structures. It doesn't require users to specify how data is stored or understand the specific data storage methods. Therefore, different database systems with completely different underlying structures can use the same SQL as an interface for data input and management. SQL statements can be nested, giving it great flexibility and powerful functionality. Large Language Models (LLMs) are deep learning models trained on large amounts of text data that can generate natural language text or understand the meaning of language text. LLMs can handle various natural language tasks, such as text classification, question answering, and dialogue, and are an important pathway to artificial intelligence. Currently, LLMs use a similar Transformer architecture and pre-training objectives (such as Language Modeling) to small models, differing only in increased model size, training data, and computational resources.

[0003] In existing technologies, considering the widespread application of SQL and the inherent complexity of SQL statements, there are already technical solutions based on large language models to assist personnel in generating SQL statements. For example, Chinese patent CN202411139469.3 generates a method and device for generating SQL statements based on a large language model. In the process of generating SQL statements across multiple dialogue rounds, if the received text to be converted is from a non-first dialogue round in the current dialogue segment, in order to generate accurate SQL statements based on previous historical dialogues, historical texts to be converted and historical SQL statements are obtained. The large language model then determines the distinguishing fields based on the text to be converted and the historical texts to be converted. According to the SQL statement field modification rules and data table structure information, the target fields of the distinguishing fields in the historical SQL statements are modified to obtain the target SQL statement. This avoids the large language model generating SQL statements based on complex text, thus improving the accuracy and efficiency of SQL statement generation.

[0004] However, in actual implementation, the inventors found that because large language models have a larger model scale than traditional artificial intelligence models, they require a lot of computing power, especially during the training phase. Under the premise of limited computing power, it is impossible to use the large models that have the better performance mentioned above. Small and medium-sized enterprises have to choose some relatively small models. Due to the limitations of small models in terms of the number of parameters and model capacity, when the task becomes more complex and requires the model to have a deep understanding and reasoning ability, the performance of small models is far inferior to that of large models. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, a SQL retrieval method based on a small parameter model is provided.

[0006] The specific technical solution is as follows:

[0007] A SQL retrieval method based on a small parameter model, comprising:

[0008] Step S1: Optimize and adjust the parameters of the first model to form a small model for the vertical domain, and form a knowledge graph for the database to be searched;

[0009] Step S2: Obtain the user's search requirements, and input prompt words into the vertical domain small model and the knowledge graph based on the search requirements to obtain the first SQL statement output by the vertical domain small model and the second SQL statement output by the knowledge graph;

[0010] Step S3: Input the first SQL statement and the second SQL statement into the second model to obtain the third SQL statement output by the second model for retrieval.

[0011] On the other hand, in step S1, the model generation method for the vertical domain small model includes:

[0012] Step A1: Perform model quantization on the first model to obtain a quantized model;

[0013] Step A2: Train the quantization model using an optimized dataset associated with the database to be retrieved;

[0014] Step A3: During the training process, LoRA fine-tuning is performed on the quantization model to obtain the small model for the vertical domain.

[0015] On the other hand, step A1 includes:

[0016] Step A11: Quantize the weights and calculations of the first model from FP32 to FP16 to obtain a pre-quantized model;

[0017] Step A12: Map the model parameters of the pre-quantized model to low-precision integers to obtain the quantized model.

[0018] On the other hand, prior to performing step A1, a dataset construction process for generating the optimized dataset is also included, the dataset construction process including:

[0019] Step B1: Obtain a general dataset, extract elements from the general dataset to obtain the elements to be screened;

[0020] Step B2: Verify the elements to be screened to obtain the elements to be screened;

[0021] Step B3: Perform sample amplification based on the screened elements to construct the optimized dataset.

[0022] On the other hand, step B2 includes:

[0023] Step B21: Deduplicate and perform meaning checks on the elements to be screened to obtain the first screening element;

[0024] Step B22: Perform syntactic optimization on the first screening element to obtain the screening element.

[0025] On the other hand, in step S1, the knowledge graph construction method of the knowledge graph includes:

[0026] Step C1: Extract table structure information from the database to be searched;

[0027] Step C2: Generate graph nodes based on multiple fields in the table structure information and the database table;

[0028] Step C3: Connect the graph nodes according to the foreign key relationships in the table structure information to form the knowledge graph.

[0029] On the other hand, step S2 includes:

[0030] Step S21: Classify according to the search requirements to determine the search type;

[0031] Step S22: Invoke the corresponding dynamic template according to the search type, and generate the prompt words according to the dynamic template and the search requirements;

[0032] Step S23: Input prompt words into the vertical domain mini-model and the knowledge graph respectively to obtain the first SQL statement and the second SQL statement.

[0033] On the other hand, in step S22, the prompt words are also added based on the table structure information of the database to be searched.

[0034] The above technical solution has the following advantages or beneficial effects:

[0035] To address the issue of poor SQL statement processing performance when deploying small models under low computing power conditions in existing technologies, this embodiment first optimizes and adjusts the parameters of the first model with fewer parameters, forming a small model for a vertical domain, and configures a knowledge graph corresponding to the database to be retrieved to form auxiliary information. In subsequent processing, SQL statements and corresponding auxiliary information can be obtained by querying the small model for the vertical domain and the knowledge graph, respectively, and then synthesized by the second model, achieving accurate generation of SQL statements under low-parameter models. Attached Figure Description

[0036] Embodiments of the invention will be described more fully with reference to the accompanying drawings. However, the drawings are for illustration and explanation only and do not constitute a limitation on the scope of the invention.

[0037] Figure 1 This is an overall schematic diagram of an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the model generation method in an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of step A1 in an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the dataset construction process in an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of step B2 in an embodiment of the present invention;

[0042] Figure 6 This is a schematic diagram of the knowledge graph construction method in an embodiment of the present invention;

[0043] Figure 7 This is a schematic diagram of step S2 in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0046] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0048] This invention includes:

[0049] A SQL retrieval method based on a small parameter model, such as Figure 1 As shown, it includes:

[0050] Step S1: Optimize and adjust the parameters of the first model to form a small model for the vertical domain, and form a knowledge graph for the database to be searched;

[0051] Step S2: Obtain the user's search requirements, and input prompt words into the vertical domain small model and the knowledge graph based on the search requirements to obtain the first SQL statement output by the vertical domain small model and the second SQL statement output by the knowledge graph.

[0052] Step S3: Input the first SQL statement and the second SQL statement into the second model to obtain the third SQL statement output by the second model for retrieval.

[0053] Specifically, in response to the problem that the processing effect of small models under low computing power conditions in the existing technology is poor, this embodiment first optimizes and adjusts the parameters of the first model with fewer parameters for the database to be retrieved, forming a small model of the vertical domain, and configures a knowledge graph corresponding to the database to be retrieved to form auxiliary information.

[0054] In subsequent processing, SQL statements and corresponding auxiliary information can be obtained by asking questions to the small model of the vertical domain and the knowledge graph, respectively. Then, they are synthesized by the second model, which realizes the accurate generation of SQL statements under the low-parameter model.

[0055] In practice, the above-mentioned SQL retrieval method is mainly configured as a software implementation in a computer system, especially an SQL database system.

[0056] The first model is the TEXT2SQL model, preferably the SQLCoder-7B-2 model. It is used as the basic model of the system.

[0057] Based on the first model, model quantization and parameter tuning are performed to obtain a small vertical domain model optimized for the database to be retrieved, thereby significantly reducing memory usage and computational complexity and ensuring model performance at low precision.

[0058] Accordingly, considering the low accuracy of small models, this embodiment also introduces a correction process based on knowledge graphs.

[0059] Among them, the knowledge graph is a graph constructed after parsing the database to be retrieved, which stores a large amount of information such as fields, keys and table structure related to the database to be retrieved.

[0060] After constructing the knowledge graph, a GraphRAG model is attached to it, which can also output SQL statements through interaction with a large language model.

[0061] The second model is a general-domain model that receives the outputs of the first model and the knowledge graph respectively, and cross-checks the outputs of the two to piece together the final result.

[0062] By splitting the traditional large language model based on a single model into three smaller models, the overall number of layers in the model can be effectively reduced and the gradient explosion problem during training of models with excessively deep layers can be avoided. This significantly reduces the demand for computing power. At the same time, the output information of the smaller models is verified through a knowledge graph-based verification mechanism, which ensures the accuracy of the output sentences.

[0063] In one embodiment, such as Figure 2 As shown, in step S1, the model generation method for the small model of the vertical domain includes:

[0064] Step A1: Quantize the first model to obtain a quantized model;

[0065] Step A2: Train the quantization model using an optimized dataset associated with the database to be retrieved;

[0066] Step A3: Fine-tune the quantized model using LoRA during training to obtain a small model for the vertical domain.

[0067] Specifically, to achieve good output accuracy under small model conditions, this embodiment first quantizes the first model to obtain a quantized model, reducing the model size while maintaining a certain level of output accuracy. Then, an optimized dataset associated with the database to be retrieved is constructed. The quantized model is trained based on this optimized dataset, and local parameters of the quantized model are adjusted using LoRA fine-tuning technology during training, thereby obtaining a small model corresponding to the vertical domain of the database to be retrieved. The introduction of LoRA fine-tuning technology further reduces the computational power requirement.

[0068] Specifically, in the training loop, each batch of data is processed as follows: First, the gradients are zeroed out, forward propagation is performed to calculate the output and loss, followed by backpropagation to calculate the gradients. Then, gradient clipping is used to prevent gradient explosion, and finally, the model parameters are updated. The specific fine-tuning parameters are as follows:

[0069] Rank: The rank of LoRA is set to 16 to ensure that the model can capture the key structured information in the Text2SQL task.

[0070] Learning rate: Set a learning rate of 1e4 for the LoRA part to ensure a smooth update process and avoid excessive perturbation of the pre-trained weights.

[0071] Regularization: L2 regularization is used in combination with gradient clipping (1.0) to prevent overfitting in low-resource scenarios.

[0072] In one embodiment, such as Figure 3 As shown, step A1 includes:

[0073] Step A11: Quantize the weights and calculations of the first model from FP32 to FP16 to obtain a pre-quantized model;

[0074] Step A12: Map the model parameters of the pre-quantized model to low-precision integers to obtain the quantized model.

[0075] Specifically, to achieve better quantization results for the model, this embodiment first employs a calibration method combining dynamic and static quantization to quantize the model's weights and calculations from FP32 to FP16. Then, INT8 quantization technology is used to map the model parameters to low-precision integers, thereby reducing the overall data volume of the model and obtaining a quantized model.

[0076] In one embodiment, prior to performing step A1, a dataset building process for generating the optimized dataset is also included, such as... Figure 4 As shown, the dataset construction process includes:

[0077] Step B1: Obtain a general dataset, extract elements from the general dataset to obtain the elements to be screened;

[0078] Step B2: Verify the elements to be screened to obtain the screening elements;

[0079] Step B3: Expand the sample based on the screening elements to build an optimized dataset.

[0080] Specifically, in order to achieve better optimization results for the model, this embodiment also pre-constructs a dataset construction process corresponding to the database retrieval process.

[0081] Specifically, the process begins by obtaining a general dataset, extracting elements from the dataset, and finally identifying the elements to be screened.

[0082] This general dataset corresponds to the design of the database to be retrieved. It contains examples of multiple fields that may appear in the database to be retrieved, as well as the SQL statements that can be used.

[0083] The above content is referred to as the element to be screened. For the element to be screened, invalid content, including redundancy, errors, ambiguity, etc., needs to be removed to obtain the accurate element to be screened.

[0084] Finally, sample augmentation is used to construct an optimized dataset to improve the generalization ability of the trained model.

[0085] In one embodiment, such as Figure 5 As shown, step B2 includes:

[0086] Step B21: Remove duplicates and perform meaning checks on the elements to be screened to obtain the first screening element;

[0087] Step B22: Perform syntactic optimization on the first screening element to obtain the screening element.

[0088] Specifically, to achieve effective screening of each element, in this embodiment, the elements to be screened are first deduplicated and their meanings are checked to obtain the first screening element, including:

[0089] Inaccurate labels, redundant SQL queries, or semantic ambiguities can be removed through both automatic and manual methods. Specifically, this may include the following steps: First, an automated script sequentially performs semantic recognition on the elements to be screened.

[0090] Once similar semantics are identified, the corresponding elements are extracted and compared. If identical elements are found, one of them is removed, thus eliminating redundant SQL queries.

[0091] When inaccurate or incomprehensible semantics are identified, the corresponding elements are extracted and verified, including simulating the execution of the corresponding SQL statements. If the execution fails, it indicates that the data may be incorrect and needs to be removed.

[0092] When there is a conflict between the semantics identified for an element and the semantics identified for a tag, it indicates that the tag annotation may be inaccurate. In this case, an automated script is used to regenerate the corresponding tag based on the element.

[0093] Based on this, especially after removing duplicate and redundant elements, the existing dataset labels were re-examined and re-labeled to ensure that each SQL element has a clear semantic reference.

[0094] Following the above verification process, in order to improve the model's generation efficiency, while keeping the query semantics unchanged, various forms of natural language queries are generated by modifying the syntactic structure and changing the word order.

[0095] In one embodiment, such as Figure 6 As shown, in step S1, the knowledge graph construction method includes:

[0096] Step C1: Extract table structure information from the database to be retrieved;

[0097] Step C2: Generate graph nodes based on multiple fields in the table structure information and the database table;

[0098] Step C3: Connect the graph nodes according to the foreign key relationships in the table structure information to form a knowledge graph.

[0099] Specifically, to achieve effective generation of the knowledge graph, this embodiment first extracts the table structure information from the database to be retrieved. Typically, a database system has multiple database tables, each containing corresponding fields and rows.

[0100] The above content exhibits a hierarchical structure and interrelationships. Graph nodes are generated based on multiple fields in the table structure information and the database table. Finally, these graph nodes are connected according to the foreign key relationships in the table structure information to form a knowledge graph.

[0101] In actual retrieval processes, the model can retrieve these relationships when generating SQL, thereby generating more accurate SQL queries, especially complex multi-table join queries. Furthermore, GraphRAG provides additional background information to the model generation process by retrieving relevant nodes from the knowledge graph. This information helps the model better understand the mapping relationships between fields.

[0102] In one embodiment, such as Figure 7 As shown, step S2 includes:

[0103] Step S21: Classify according to search needs to determine the search type;

[0104] Step S22: Call the corresponding dynamic template according to the search type, and generate prompt words based on the dynamic template and search requirements;

[0105] Step S23: Input prompt words into the vertical domain small model and the knowledge graph respectively to obtain the first SQL statement and the second SQL statement.

[0106] Specifically, in order to achieve better search results, in this embodiment, the search requirements are first classified to determine the search type, such as "SELECT", "INSERT", "UPDATE", etc. Then, according to the search type, the corresponding dynamic template can be called to correspond to different operations. Based on the dynamic templates of different operations, prompt words are generated by filling them.

[0107] In addition, in step S22, prompt words are added based on the table structure information of the database to be searched.

[0108] Specifically, in order to achieve a better prompting effect, this embodiment designs three dimensions for auxiliary description when constructing prompt words, including the type of retrieval operation mentioned above, as well as database structure and complexity.

[0109] For the database structure, table structure information is pre-extracted to explicitly include the database table structure and field information in the prompts. For example, table field names and types are directly embedded into the input query, helping the model to more clearly understand the context of the SQL query to be generated.

[0110] To address complexity, a variable dynamic template was designed. In simple queries, the suggestion words can be relatively concise, while in complex queries such as multi-table joins, more information on inter-table relationships and field mappings is added to the suggestion words to ensure that the model can correctly parse the query intent.

[0111] Based on the above design, prompt words are generated and then input into the first model and the knowledge graph, respectively. The knowledge graph is mounted under a GraphRAG model, which can retrieve nodes in the knowledge graph based on the input prompt words, providing additional background information for the model generation process. This information can help the model better understand the mapping relationships between fields.

[0112] To verify the effectiveness of the above method, the inventors also conducted an experiment.

[0113] This experiment uses the open-source AI native data application development framework DB-GPT to demonstrate the results, and the processed models are placed in the / DB-GPT / models path.

[0114] In this experiment, after generating the vertical category model and knowledge graph by completing the above five steps in advance using the method of this invention, multiple questions were posed to both the system using the method of this invention and the system not using the method of this invention, with the same questions being asked in both cases (and typos were intentionally entered during the questioning process). The response results, system time consumption, and system memory usage were recorded after multiple questions. Statistical analysis was performed on the SQL generation accuracy, model inference speed, and GPU memory consumption under the two forms to obtain the final experimental results and conclusions.

[0115] Experimental results:

[0116]

[0117]

[0118] Table 1

[0119] Experimental conclusions

[0120] Accuracy: On datasets such as Spider and WikiSQL, the quantized model, after combining prompt words and using knowledge graphs, performs close to the performance of the large unquantized model, with accuracy loss controlled within 5%.

[0121] Inference speed: Using INT8 quantization improves inference speed by 3x. However, in FP16 mode, although the accuracy is higher, the inference speed is still slightly slower than INT8. INT8 quantization has greater advantages in computational efficiency and memory bandwidth, so it is generally faster than FP16 during inference.

[0122] Resource consumption: Fine-tuning using LoRA reduces the required GPU memory by approximately 50%. Quantization to INT8 significantly reduces memory usage, bandwidth pressure, and computational burden during inference, substantially lowering hardware resource requirements and making it suitable for deployment scenarios with limited computing power.

[0123] Those skilled in the art will understand that various aspects, or possible implementations of various aspects, of the present invention can be embodied as systems, methods, or computer program products. Therefore, various aspects, or possible implementations of various aspects, of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, etc.), or embodiments combining software and hardware aspects, all collectively referred to herein as "circuit," "module," or "system." Furthermore, various aspects, or possible implementations of various aspects, of the present invention can take the form of computer program products, which are computer instructions stored in memory.

[0124] The memory can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable read-only memory (CD-ROM).

[0125] A processor in a computer reads computer instructions stored in memory, enabling the processor to execute the functional actions specified in each step or combination of steps in a flowchart; and to generate means for implementing the functional actions specified in each block or combination of blocks in a flowchart.

[0126] It should be understood that a processor in a computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components used to execute the aforementioned computer instructions.

[0127] Computer instructions may be executed entirely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or entirely on a remote computer or server. It should also be noted that in some alternative implementations, the functions indicated by the steps in the flowchart or the blocks in the block diagram may not occur in the order shown in the diagram. For example, depending on the functions involved, two consecutive steps or blocks may actually be executed approximately simultaneously, or these blocks may sometimes be executed in reverse order.

[0128] Of course, in practical applications, the various components of a computer system are coupled together through a bus system. The bus system is used to enable communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0129] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A SQL retrieval method based on a small parameter model, characterized in that: include: Step S1: Optimize the model and adjust the parameters of the first model to form a small model for the vertical category, and form a knowledge graph for the database to be searched; Step S2: Obtain the user's search requirements, and input prompt words to the vertical category domain small model and the knowledge graph respectively based on the search requirements to obtain a first SQL statement output by the vertical category domain small model and a second SQL statement output by the knowledge graph; Step S3: Input the first SQL statement and the second SQL statement into the second model to obtain a third SQL statement output by the second model for retrieval.

2. The SQL search method according to claim 1, characterized in that: In step S1, the model generation method of the vertical domain small model includes: Step A1: performing model quantization on the first model to obtain a quantized model; Step A2: training the quantitative model using an optimized data set associated with the database to be searched; Step A3: During the training process, the quantization model is fine-tuned by LoRA to obtain the small model of the vertical domain.

3. The SQL search method according to claim 2, characterized in that: The step A1 comprises: Step A11: quantizing the weights and calculations of the first model from FP32 to FP16 to obtain a pre-quantized model; Step A12: Mapping the model parameters of the pre-quantization model to low-precision integers to obtain the quantization model.

4. The SQL search method according to claim 2, characterized in that: Before executing the step A1, a data set construction process for generating the optimized data set is also included, and the data set construction process includes: Step B1: obtaining a general data set, extracting elements from the general data set, and obtaining elements to be screened; Step B2: checking the elements to be screened to obtain screening elements; Step B3: Perform sample amplification according to the screening elements to construct the optimized data set.

5. The SQL search method according to claim 4, characterized in that: The step B2 comprises: Step B21: removing duplication and checking the meaning of the elements to be screened to obtain a first screening element; Step B22: grammatically optimize the first screening element to obtain the screening element.

6. The SQL search method according to claim 1, characterized in that: In step S1, the knowledge graph construction method of the knowledge graph includes: Step C1: extracting table structure information from the database to be searched; Step C2: Generate graph nodes according to the multiple fields and database tables in the table structure information; Step C3: Connect the graph nodes according to the foreign key relationship in the table structure information to form the knowledge graph.

7. The SQL search method according to claim 1, characterized in that: The step S2 comprises: Step S21: classify the search requirements to determine the search type; Step S22: calling a corresponding dynamic template according to the search type, and generating the prompt word according to the dynamic template and the search requirement; Step S23: input prompt words into the vertical domain small model and the knowledge graph respectively to obtain the first SQL statement and the second SQL statement.

8. The SQL search method according to claim 7, characterized in that: In the step S22, the prompt word is also added according to the table structure information of the database to be searched.

Images