Structural perception parameter efficient fine tuning method and system for cross-domain Text-to-SQL task
By preprocessing and encoding natural language problems and database structures, and building a fine-tuning language model, the problems of excessive parameters and poor generalization in the existing methods are solved, and efficient cross-domain Text-to-SQL tasks under different hardware memory conditions are realized, and more accurate SQL query statements are generated.
Patent Information
- Application Number
- CN202510341065.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-21
- Publication Date
- 2025-07-18
AI Technical Summary
When generating SQL query statements, the existing cross-domain Text-to-SQL methods have problems such as excessive parameters, high computational cost and poor generalization. Especially when dealing with complex queries, it is difficult to effectively capture the complex relationship between natural language problems and database structure.
By obtaining natural language problems, SQL statements and database structure information, preprocessing, constructing serialized input and graph structures, using self-attention mechanism and relational graph neural network for encoding, integrating semantic modeling and structural modeling coding, building fine-tuning language models, and adapting to devices with different hardware memory sizes for efficient fine-tuning.
It realizes that when most of the model parameters are frozen, the complex semantic and structural relationship between natural language problems and database information is captured lightly, and more accurate SQL query statements are generated. It is suitable for a variety of cross-domain database scenarios, avoiding the limitations of a large number of parameter adjustments.
Smart Images
Figure CN120336364A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of file management, and more specifically, to a method and system for efficiently fine-tuning structure-aware parameters for cross-domain Text-to-SQL tasks. Background Art
[0002] Relational database software has gradually become the foundation and core of major information systems on the Internet due to its structured and efficient data management method. Structured Query Language (SQL) is a query language widely used in current relational databases for quickly retrieving data in the database. However, for ordinary users in non-computing technical fields, it is somewhat difficult and has a learning threshold to understand and master this structured query language, making it difficult to quickly obtain valuable information in the database. Therefore, in order to improve the usability and ease of use of the database, the cross-domain Text-to-SQL task has been proposed. Its purpose is to automatically convert a natural language question that can express the user's intention into a semantically corresponding SQL query statement under the condition of a given database structure. This technology enables users to quickly obtain valuable information in the database only through natural language, thereby assisting in decision-making.
[0003] Existing cross-domain Text-to-SQL methods mainly include template rule-based methods, traditional graph neural network-based methods, and pre-trained language model-based methods. Although these methods can all generate SQL instructions corresponding to questions, they have limitations. Template rule-based methods rely on a large amount of manual feature engineering and specific domain knowledge, lacking the adaptability and scalability required to handle diverse and complex problems; traditional graph neural network-based methods, although effectively capturing the relationship between questions and database structures using graph structures, are limited by the local information aggregation mechanism when dealing with complex queries, making it difficult to effectively model complex relationship links and lacking the support of large-scale corpora possessed by pre-trained language models; pre-trained language model-based methods, although showing strong semantic understanding capabilities, do not make good use of the structure-aware advantages brought by traditional graph design networks and require gradient updates for all parameters of the original language model. Currently, the mainstream Graphix-T5 method in this field can be regarded as an improved method that combines traditional graph neural networks and pre-trained language models, but such methods introduce a large number of additional parameters for structure awareness on the basis of the original language model, resulting in the need to perform full-scale gradient updates on the huge parameters of the entire model, with the significant defects of excessive training parameters and excessive computational costs. Summary of the Invention
[0004] The purpose of the present invention is to disclose a method and system for efficiently fine-tuning structure-aware parameters for cross-domain Text-to-SQL tasks with better effects.
[0005] To achieve the above object, the present invention provides a structure-aware parameter-efficient fine-tuning method for cross-domain Text-to-SQL tasks, including:
[0006] Obtain data pairs composed of natural language questions and SQL statements, as well as database structure information corresponding to the data pairs;
[0007] Preprocess the data pairs and database structure information to construct a serialized input and a graph structure containing node and edge relationships;
[0008] Perform semantic modeling encoding and structural modeling encoding on the serialized input and the graph structure respectively;
[0009] Fuse the semantic modeling encoding and the structural modeling encoding to construct a fine-tuned language model, and obtain a trained SQL model according to the fine-tuned language model;
[0010] Input the natural language question to be answered and the corresponding database structure information into the SQL generation model to obtain an SQL query statement.
[0011] Further, preprocessing the data pairs and database structure information specifically includes:
[0012] Perform unified stop-word processing, word segmentation processing, and standardized indentation processing on the natural language questions in the data pairs to obtain a natural language question word sequence;
[0013] Perform format normalization and keyword normalization processing on the SQL statements in the database to obtain a standard and normalized SQL;
[0014] Parse the database information, extract all table names and corresponding column names included in the database, as well as the primary key and foreign key information of all tables, and directly splice them to obtain a database information word sequence.
[0015] Further, constructing a serialized input and a graph structure containing node and edge relationships includes:
[0016] Splice the natural language question sequence with each word in the database information words using an interval symbol | to obtain a complete serialized input;
[0017] Use the natural language question sequence and each word in the database information words as nodes of the graph structure, introduce a predefined relationship prior matching rule, and determine the edge type of the graph structure according to the matching degree between the question words and the database words, to obtain a graph structure containing nodes and edges composed of question words and database words.
[0018] Further, performing semantic modeling encoding and structural modeling encoding on the serialized input and the graph structure respectively specifically includes:
[0019] Model and encode the serialized input using the self-attention mechanism, aggregate the representation information of the question words and database words through the self-attention network, and obtain the semantic model encoding;
[0020] Use the obtained semantic model encoding as the initial representation of the graph structure, and perform structure encoding on the graph structure through the relational graph neural network to obtain the structure model encoding.
[0021] Furthermore, a fine-tuned language model is constructed. The method for obtaining the trained SQL model according to the fine-tuned language model is as follows: for devices with a small hardware video memory size, use the parameter-efficient fine-tuning method based on structured prefix tuning to fine-tune and train the language model;
[0022] For devices with a large hardware video memory size, use the parameter-efficient fine-tuning method based on structured bypass adapter tuning to fine-tune and train the language model.
[0023] Furthermore, for devices with a small hardware video memory size, fuse the semantic model encoding and the structure model encoding to construct a structure-aware prefix language model, specifically including:
[0024] Construct the semantic model encoding and the structure model encoding by direct addition to obtain the structure-aware prefix language model.
[0025] Furthermore, for devices with a small hardware video memory size, based on the structure-aware prefix language model, obtain the SQL model, specifically including:
[0026] Perform parameter-efficient fine-tuning training on the language model based on the structure-aware prefix language model. While freezing most of the model's parameters unchanged, perform gradient updates on the parameters introduced by the structure-aware prefix language model to obtain the SQL model.
[0027] Furthermore, for devices with a large hardware video memory size, fuse the semantic model encoding and the structure model encoding to construct a structure-aware adapter language model, specifically including:
[0028] Construct the semantic model encoding and the structure model encoding by adding representations based on a gating function to obtain a bypass structure-aware adapter language model.
[0029] Furthermore, based on the bypass structure-aware adapter language model, obtain the SQL model, specifically including:
[0030] Based on the bypass structure-aware adapter language model, the language model is trained with parameter-efficient fine-tuning. While most of the model's parameters are frozen, the parameters of the native adapter and the introduced bypass structure-aware adapter language model are updated by gradient to obtain the SQL model.
[0031] In addition, the present invention also provides a structure-aware parameter-efficient fine-tuning system for cross-domain Text-to-SQL tasks, which is characterized by including:
[0032] An acquisition module: acquiring data pairs composed of natural language questions and SQL statements, as well as database structure information corresponding to the data pairs;
[0033] A preprocessing module: preprocessing the data pairs and database structure information to construct serialized inputs and graph structures including node and edge relationships;
[0034] An encoding module: performing semantic modeling encoding and structure modeling encoding on the serialized inputs and graph structures respectively;
[0035] A model module: adopting different structure-aware parameter-efficient fine-tuning methods for devices with different hardware video memory sizes to obtain an SQL generation model;
[0036] A generation module: inputting the natural language question to be answered and the corresponding database structure information into the SQL generation model to obtain an SQL query statement.
[0037] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:
[0038] By integrating semantic modeling encoding and structure modeling encoding, the present invention effectively captures the complex semantic and structural relationships between natural language questions and database information, effectively ensuring the generation of more accurate SQL query statements. By fine-tuning the structure-aware prefix language model or the structure-aware bypass adapter, the present invention realizes structure awareness of the language model in a lightweight manner while freezing most of the model's parameters. At the same time, the present invention is applicable to different hardware video memory sizes and various cross-domain database scenarios, thereby avoiding the limitations of the existing methods that require adjusting a large number of model parameters and have poor generalization. Description of the Drawings
[0039] Figure 1 It is a flowchart of the structure-aware parameter-efficient fine-tuning method for cross-domain Text-to-SQL tasks described in Embodiment 1;
[0040] Figure 2 It is a framework diagram of the fine-tuning method based on the structure-aware prefix language model described in Embodiment 3;
[0041] Figure 3It is a framework diagram of the method for fine-tuning the adapter language model based on the bypass structure perception described in Embodiment 3;
[0042] Figure 4 It is a block diagram of the structure-aware parameter-efficient fine-tuning system for cross-domain Text-to-SQL tasks described in Embodiment 4; Detailed implementation manners
[0043] The accompanying drawings are only for illustrative purposes and should not be construed as limitations to this patent;
[0044] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0045] Embodiment 1:
[0046] This embodiment provides structure-aware parameter-efficient fine-tuning for cross-domain Text-to-SQL tasks as shown in Figure 1 and includes:
[0047] Obtain data pairs composed of natural language questions and SQL statements and database structure information corresponding to the data pairs;
[0048] Preprocess the data pairs and database structure information to construct serialized inputs and graph structures containing node and edge relationships;
[0049] Perform semantic modeling encoding and structure modeling encoding on the serialized inputs and graph structures respectively;
[0050] Fuse the semantic modeling encoding and the structure modeling encoding to construct a fine-tuning language model, and obtain a trained SQL model according to the fine-tuning language model;
[0051] Input the natural language question to be answered and the corresponding database structure information into the SQL generation model to obtain an SQL query statement.
[0052] In this embodiment, by fusing semantic modeling encoding and structure modeling encoding, the complex semantic and structural relationships between natural language questions and database information are effectively captured, effectively ensuring the generation of more accurate SQL query statements. The present invention realizes structure-aware of the language model in a lightweight manner by fine-tuning the structure-aware prefix language model or the structure-aware bypass adapter while freezing most of the model parameters. At the same time, the present invention is applicable to different hardware video memory sizes and various cross-domain database scenarios, thus avoiding the limitations of the existing methods that require adjusting a large number of model parameters and have poor generalization.
[0053] Embodiment 2:
[0054] This embodiment further discloses on the basis of Embodiment 1:
[0055] Further, preprocess the data pairs and database structure information to construct a serialized input and a graph structure containing node and edge relationships, specifically including:
[0056] Perform unified stop word processing, word segmentation processing, and standardized indentation processing on the natural language questions in the data pairs to obtain a natural language question word sequence;
[0057] Normalize the format and keywords of the SQL statements in the database to obtain a standard and normalized SQL;
[0058] Parse the database information, extract all table names, corresponding column names, and primary key and foreign key information of all tables in the database, and directly splice them to obtain a database information word sequence.
[0059] Further, constructing a serialized input and a graph structure containing node and edge relationships includes:
[0060] Splice the natural language question sequence with each word in the database information words using the delimiter | to obtain a complete serialized input;
[0061] Take the natural language question sequence and each word in the database information words as nodes of the graph structure, introduce a predefined relationship prior matching rule, and determine the edge type of the graph structure according to the matching degree between the question words and the database words, obtaining a graph structure containing nodes and edges composed of question words and database words.
[0062] Further, perform semantic modeling encoding and structural modeling encoding on the serialized input and the graph structure respectively, specifically including:
[0063] Use the self-attention mechanism to perform modeling encoding on the serialized input, and aggregate the representation information of the question words and the database words through the self-attention network to obtain semantic modeling encoding;
[0064] Use the obtained semantic modeling encoding as the initial representation of the graph structure, and perform structural encoding on the graph structure through a relational graph neural network to obtain structural modeling encoding.
[0065] Further, the method for constructing a fine-tuned language model and obtaining a trained SQL model according to the fine-tuned language model is as follows:
[0066] For devices with a relatively small hardware video memory size, use a parameter-efficient fine-tuning method based on structured prefix tuning to fine-tune and train the language model;
[0067] For devices with a relatively large hardware video memory size, use a parameter-efficient fine-tuning method based on structured bypass adapter tuning to fine-tune and train the language model.
[0068] Further, for devices with a relatively small hardware video memory size, fuse the semantic modeling encoding and the structural modeling encoding to construct a structure-aware prefix language model, specifically including:
[0069] Construct the structure-aware prefix language model by directly adding the semantic modeling encoding and the structural modeling encoding.
[0070] Further, for devices with a relatively small hardware video memory size, obtain an SQL model based on the structure-aware prefix language model, specifically including:
[0071] Perform parameter-efficient fine-tuning training on the language model based on the structure-aware prefix language model. While freezing most of the model's parameters unchanged, perform gradient updates on the parameters introduced by the structure-aware prefix language model to obtain the SQL model.
[0072] Further, for devices with a relatively large hardware video memory size, fuse the semantic modeling encoding and the structural modeling encoding to construct a structure-aware adapter language model, specifically including:
[0073] Construct the bypass structure-aware adapter language model by adding the semantic modeling encoding and the structural modeling encoding in a manner characterized by a gating function.
[0074] Further, obtain an SQL model based on the bypass structure-aware adapter language model, specifically including:
[0075] Perform parameter-efficient fine-tuning training on the language model based on the bypass structure-aware adapter language model. While freezing most of the model's parameters unchanged, perform gradient updates on the parameters of the native adapter and the introduced bypass structure-aware adapter language model to obtain the SQL model.
[0076] In this embodiment, by fusing the semantic modeling encoding and the structural modeling encoding, the complex semantic and structural relationships between natural language questions and database information are effectively captured, effectively ensuring the generation of more accurate SQL query statements. The present invention realizes structure awareness of the language model in a lightweight manner by fine-tuning the structure-aware prefix language model or the structure-aware bypass adapter while freezing most of the model's parameters. At the same time, the present invention is applicable to different hardware video memory sizes and various cross-domain database scenarios, thus avoiding the limitations of the existing methods that require adjusting a large number of model parameters and have poor generalization.
[0077] Embodiment Three:
[0078] This embodiment further discloses on the basis of Embodiment Two:
[0079] In an optional embodiment, common natural language toolkits NLTK and Stanza are used to perform unified stop word processing, word segmentation processing, and standardized indentation processing on the natural language questions in the data pairs, obtaining a natural language question word sequence Q = q1, …, q |Q| ;
[0080] The SQL statements in the database are processed for format standardization and keyword standardization through regular expressions and rule matching, obtaining a standard specification SQL y;
[0081] The structure information S of the database D in the database file corresponding to each natural language question in the data pair is parsed, and all table names T = {t1, t2, …, t N} and the column names corresponding to each table as well as the primary key and foreign key information of all tables are extracted. Among them, all column names belonging to each table name are separated by English commas, and each table is separated by the symbol |, obtaining a database information word sequence
[0082] The natural language question sequence Q, the database name D, and the database information word sequence S are concatenated using the special symbol |, and a symbol * is concatenated at the end of the sequence, obtaining a complete serialized input sequence:
[0083]
[0084] Each word in the natural language question word sequence Q and the database information word sequence S is used as a node of the graph structure, obtaining a node set V. A predefined relationship edge set R is introduced to represent the matching relationship types between different nodes, which altogether includes three types of predefined relationship edges: question structure relationship edge R Q , database schema structure relationship edge R S and question-database link structure relationship edge For the node set V, the predefined relationship edge types are matched using a string matching algorithm, obtaining an edge set E. For example, for the question word "book" and the database table name "book", a complete match question-database link structure relationship edge is obtained; for another example, for the question word "book" and the database table name "book_name", a partial match question-database link structure relationship edge is obtained; through the above relationship edge matching process, a structure-aware prior is introduced, obtaining a graph structure G = <V, E>;
[0085] Semantic modeling encoding and structure modeling encoding are respectively performed on the serialized input and the graph structure;
[0086] Semantic encoding modeling is performed on the serialized input
[0087] In an alternative embodiment, the self-attention mechanism is used to model and encode the serialized input sequence X, and the self-attention network aggregates the representation information of the question words and database words to obtain the semantic modeling encoding. The update method is as follows:
[0088]
[0089] Perform structural modeling encoding on the graph structure
[0090] In an alternative embodiment, the obtained semantic modeling encoding is used as the initial representation of the graph structure, and the graph structure is structurally encoded through a relational graph neural network to obtain the structural modeling encoding. The update method is as follows:
[0091]
[0092] In Embodiment 1, different structure-aware parameter efficient fine-tuning methods are determined according to the hardware video memory size. Specifically:
[0093] For devices with a relatively small hardware video memory size, a parameter efficient fine-tuning method based on structured prefix tuning is used to fine-tune and train the language model;
[0094] For devices with a relatively large hardware video memory size, a parameter efficient fine-tuning method based on structured bypass adapter tuning is used to fine-tune and train the language model;
[0095] Fuse the semantic modeling encoding and the structural modeling encoding to construct a structure-aware prefix language model
[0096] In an alternative embodiment, the semantic modeling encoding and the structural modeling encoding are used to construct the fused structure-aware encoding SA_Prefix by addition, and are respectively concatenated with the prefixes P K and P V in the native prefix tuning method, as Figure 2 shown, to obtain the structure-aware prefix language model and The update method is as follows:
[0097]
[0098] where [;] represents the concatenation operation.
[0099] Based on the structure-aware prefix language model, a trained SQL model is obtained;
[0100] In an alternative embodiment, while most of the parameters of the language model are frozen, the language model is trained with parameter-efficient fine-tuning based on the structure-aware prefix language model, and the parameters of the introduced structure-aware prefix language model are updated by gradient to obtain a trained SQL model. The update method is as follows:
[0101]
[0102] K′ = [P K ; K]
[0103] V′ = [P V ; V]
[0104] where [;] represents a concatenation operation, and Q, K, and V represent the query vector, key vector, and value vector of the language model, respectively.
[0105] Fuse the semantic modeling encoding and the structure modeling encoding to construct a structure-aware adapter language model
[0106] In an alternative embodiment, the semantic modeling encoding and the structure modeling encoding are constructed by adding representations based on a gating function to obtain a bypass structure-aware adapter language model Adapter (l) , and the update method is as follows;
[0107]
[0108] where W down ∈ □ h×m and W up ∈ □ m×h are two low-rank matrices used to reduce the number of training parameters, m is the bottleneck dimension, σ represents the sigmoid function, and θ represents the tanh function.
[0109] Based on the bypass structure-aware adapter language model, a trained SQL model is obtained:
[0110] In an alternative embodiment, as Figure 3 shown, the bypass structure-aware adapter language model Adapter (l) is placed in the adapter bypass of each layer of the native adapter tuning method. While most of the model parameters are frozen, the parameters of the native adapter and the introduced bypass structure-aware adapter language model are updated by gradient to obtain a trained SQL model. The update method is as follows:
[0111] Adapter(H l ) = σ(H l W down )Wup +H l
[0112] where H l represents the vector output of the language model before the adapter insertion position.
[0113] Input the natural language question to be answered and the corresponding database structure information into the trained SQL generation model to obtain an SQL query statement.
[0114] In an alternative embodiment, it is only necessary to input the natural language question and the corresponding database structure information into the trained SQL generation model to obtain the corresponding SQL query statement.
[0115] Embodiment 4:
[0116] This embodiment provides a structure-aware parameter-efficient fine-tuning system for cross-domain Text-to-SQL tasks as shown in Figure 4 and includes:
[0117] Acquisition module: Acquire data pairs composed of natural language questions and SQL statements and the database structure information corresponding to the data pairs;
[0118] Preprocessing module: Preprocess the data pairs and the database structure information to construct a serialized input and a graph structure including node and edge relationships;
[0119] Encoding module: Perform semantic modeling encoding and structure modeling encoding on the serialized input and the graph structure respectively;
[0120] Model module: For devices with different hardware video memory sizes, adopt different structure-aware parameter-efficient fine-tuning methods to obtain an SQL generation model;
[0121] Generation module: Input the natural language question to be answered and the corresponding database structure information into the SQL generation model to obtain an SQL query statement.
[0122] This embodiment effectively captures the complex semantic and structural relationships between natural language questions and database information by fusing semantic modeling encoding and structure modeling encoding, effectively ensuring the generation of more accurate SQL query statements. The present invention realizes structure-aware of the language model in a lightweight manner by fine-tuning the structure-aware prefix language model or the structure-aware bypass adapter while freezing most of the model parameters. At the same time, the present invention is applicable to different hardware video memory sizes and various cross-domain database scenarios, thereby avoiding the limitations of the existing methods that require adjusting a large number of model parameters and have poor generalization.
[0123] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation manners here. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.
Claims
1. A method for efficiently fine-tuning structure-aware parameters for cross-domain Text-to-SQL tasks, characterized in that Including: Obtaining data pairs consisting of natural language questions and SQL statements, as well as database structure information corresponding to the data pairs; Preprocessing the data pairs and database structure information to construct a serialized input and a graph structure including node and edge relationships; Performing semantic modeling encoding and structural modeling encoding on the serialized input and the graph structure respectively; Fusing the semantic modeling encoding and the structural modeling encoding to construct a fine-tuned language model, and obtaining a trained SQL model according to the fine-tuned language model; Inputting the natural language question to be answered and the corresponding database structure information into the SQL generation model to obtain an SQL query statement.
2. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 1, characterized in that Preprocessing the data pairs and database structure information, specifically including: Performing unified stop word processing, word segmentation processing, and standardized indentation processing on the natural language questions in the data pairs to obtain a natural language question word sequence; Performing format normalization and keyword normalization processing on the SQL statements in the database to obtain a standard and normalized SQL; Parsing the database information, extracting all table names and corresponding column names included in the database, as well as the primary key and foreign key information of all tables, and directly concatenating them to obtain a database information word sequence.
3. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 1, characterized in that, Constructing a serialized input and a graph structure including node and edge relationships includes: Concatenating the natural language question sequence with each word in the database information words using an interval symbol | to obtain a complete serialized input; Taking the natural language question sequence and each word in the database information words as nodes of the graph structure, introducing a predefined relationship prior matching rule, and determining the edge type of the graph structure according to the matching degree of the question words and the database words, to obtain a graph structure including nodes and edges composed of the question words and the database words.
4. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 1, wherein Performing semantic modeling encoding and structural modeling encoding on the serialized input and the graph structure respectively, specifically including: Using a self-attention mechanism to perform modeling encoding on the serialized input, and aggregating the representation information of the question words and the database words through a self-attention network to obtain semantic modeling encoding; Using the obtained semantic modeling encoding as the initial representation of the graph structure, and performing structural encoding on the graph structure through a relational graph neural network to obtain structural modeling encoding.
5. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 1, wherein The method for constructing a fine-tuned language model and obtaining a trained SQL model according to the fine-tuned language model is as follows: For devices with a relatively small hardware video memory size, fusing the semantic modeling encoding and the structural modeling encoding to construct a structure-aware prefix language model; Based on the structure-aware prefix language model, obtaining a trained SQL model; For devices with a relatively large hardware video memory size, fusing the semantic modeling encoding and the structural modeling encoding to construct a structure-aware adapter language model; Based on the bypass structure-aware adapter language model, obtaining a trained SQL model.
6. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 5, characterized in that For devices with a relatively small hardware video memory size, fusing the semantic modeling encoding and the structural modeling encoding to construct a structure-aware prefix language model, specifically including: Constructing the semantic modeling encoding and the structural modeling encoding in a way of direct addition to obtain a structure-aware prefix language model.
7. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 5, characterized in that For devices with a relatively small hardware video memory size, obtaining an SQL model based on the structure-aware prefix language model, specifically including: Based on the structure-aware prefix language model, perform parameter-efficient fine-tuning training on the language model. While keeping most of the model's parameters frozen, update the gradients of the parameters introduced by the structure-aware prefix language model to obtain the SQL model.
8. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 5, wherein For devices with a relatively large hardware video memory size, fuse the semantic modeling encoding and the structure modeling encoding to construct a structure-aware adapter language model, specifically including: Construct the semantic modeling encoding and the structure modeling encoding by adding them based on a gating function representation to obtain a bypass structure-aware adapter language model.
9. The structure-aware parameter efficient fine-tuning method for cross-domain Text-to-SQL tasks according to claim 5, characterized in that Based on the bypass structure-aware adapter language model, obtain the SQL model, specifically including: Based on the bypass structure-aware adapter language model, perform parameter-efficient fine-tuning training on the language model. While keeping most of the model's parameters frozen, update the gradients of the parameters of the native adapter and the introduced bypass structure-aware adapter language model to obtain the SQL model.
10. A query system for the structure-aware parameter-efficient fine-tuning method for cross-domain Text-to-SQL tasks according to any one of claims 1 to 9, characterized in that, Including: Acquisition module: Acquire data pairs composed of natural language questions and SQL statements, as well as database structure information corresponding to the data pairs; Preprocessing module: Preprocess the data pairs and the database structure information to construct serialized inputs and a graph structure containing node and edge relationships; Encoding module: Perform semantic modeling encoding and structure modeling encoding on the serialized inputs and the graph structure respectively; Model module: For devices with different hardware video memory sizes, adopt different structure-aware parameter-efficient fine-tuning methods to obtain an SQL generation model; Generation module: Input the natural language question to be answered and the corresponding database structure information into the SQL generation model to obtain an SQL query statement.