Query statement generation method and device based on large language model, equipment and medium

By introducing multiple attention mechanisms and multiple encoders into the large language model, combining query intention recognizers and template guidance attention modules, the problem of low accuracy in query statement generation in the prior art is solved, and the complexity and Efficient processing and accurate identification of fuzzy queries.

CN118964560APending Publication Date: 2024-11-15CHINA UNIONPAY

Patent Information

Application Number
CN202411008424.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, the accuracy of query statement generation is not high, especially when handling complex and fuzzy queries, it is difficult to accurately identify key information and filter interference information.

Method used

Using a large language model based on multiple attention mechanism, by setting up multiple encoders, each encoder has an independent weight matrix, encodes the target vectors separately, generates context vectors, and generates them through context vector connections of different encoders. Query statements, combining query intention recognizers and template guidance attention modules, improve the accuracy of statement generation.

Benefits of technology

It significantly improves the accuracy and efficiency of query statement generation, can better capture complex and abstract semantic relationships, accurately distinguish information when processing fuzzy queries, and generate SQL queries that are more in line with user needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964560A_ABST
    Figure CN118964560A_ABST
Patent Text Reader

Abstract

The invention provides a query statement generation method and device based on a large language model, equipment and a medium, the large language model comprises a plurality of encoders, and the method comprises the following steps: obtaining a query text; vectorizing the query text to obtain a target vector representation corresponding to the query text; the target vector representation is input to a plurality of encoders, the target vector representation is encoded through the weight matrixes of the encoders, context vectors corresponding to the encoders are obtained, and the weight matrixes of different encoders are different; according to the context vector corresponding to each encoder, a target query statement is generated, a model is allowed to focus on different information features in different attention heads, different aspects of a query text are independently captured, and information is processed on multiple abstract levels, so that a complex and abstract semantic relationship is better captured; when complex and fuzzy questions are processed, the importance degree of the information can be well distinguished, and the accuracy of query statement generation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a query statement generation method, device, equipment and medium based on a large language model. Background Art

[0002] With the rapid development of information technology, databases have become an important part of our daily life and work. How to enable ordinary users to access and use information in databases more conveniently has become an important research topic. Text-to-SQL converts users' natural language queries into corresponding Structured Query Language (SQL) query statements through a Large Language Model (LLM), allowing users to interact with the database directly through natural language.

[0003] In related technologies, text-to-SQL is mostly processed using a general Transformer large model. When processing text to SQL tasks, the general Transformer large model encodes the natural language query text and database input by the user into an embedding vector, and uses the traditional Transformer structure for context fusion and self-attention mechanism processing to capture the relationship between the text and the database structure. Finally, a SQL query statement that conforms to the grammatical rules is generated through the decoder. During this period, the loss function is used to continuously optimize the model weights during the training process to improve the generation accuracy.

[0004] However, the accuracy of query statement generation in the prior art is not high. Summary of the invention

[0005] The present application provides a query statement generation method, apparatus, device and medium based on a large language model, thereby solving the problem of low accuracy of query statement generation in the prior art.

[0006] In a first aspect, the present application provides a query statement generation method based on a large language model, wherein the large language model includes a plurality of encoders, and the method includes:

[0007] Get the query text;

[0008] Performing vectorization processing on the query text to obtain a target vector representation corresponding to the query text;

[0009] Inputting the target vector representation to the plurality of encoders to encode the target vector representation using a weight matrix of each encoder to obtain a context vector corresponding to each encoder, wherein the weight matrices of different encoders are different;

[0010] Generate a target query statement according to the context vectors corresponding to the encoders.

[0011] The present application proposes a query statement generation method based on a large language model, which uses a large language model based on a multiple attention mechanism to generate query statements. The large language model is improved on the basis of the general Transformer large model, and multiple encoders are set. For the corresponding target vector representation of the query text, the target vector representation is encoded based on the weight matrix in each encoder through multiple encoders of the large language model, and the context vector is output by each encoder. Then, based on the context vectors output by different encoders, the target query statement is generated. Since the weight matrix in each encoder is different, each encoder can pay attention to the different semantic and structural features of the query text when extracting features, so that the model can generate specific templates in different subspaces, allowing the model to focus on different information features in different attention heads, independently capture different aspects of the query text, and process information at multiple abstract levels, so as to better capture complex and abstract semantic relationships, and can distinguish the importance of information well when dealing with complex and ambiguous questions, thereby improving the accuracy of query statement generation.

[0012] Optionally, the encoder includes a first self-attention layer and a context vector layer; the inputting the target vector representation to the multiple encoders to encode the target vector representation through a weight matrix of each encoder to obtain a context vector corresponding to each encoder, including:

[0013] The target vector representation is input into the first self-attention layer of the multiple encoders, so as to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain multiple output vectors of each encoder; the multiple output vectors of each encoder are input into the corresponding context vector layer, so as to perform weighted processing on the multiple output vectors through each context vector layer to obtain the context vector corresponding to each encoder.

[0014] Here, the encoder of the present application includes a self-attention layer for assigning different attention weights to the target vector representation so as to extract semantic information in a targeted manner, and a context vector layer for weighted processing to obtain a context vector. The self-attention layer is used to focus on different semantic features and flexibly and accurately identify multiple key information in the query text. The context vector layer can output a context vector to help the large model focus on the relevant parts of the encoder output, further improving the accuracy of query statement generation.

[0015] Optionally, before inputting the target vector representation to the first self-attention layer of the plurality of encoders to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain a plurality of output vectors of each encoder, the method further includes:

[0016] For each encoder, obtain a training sample corresponding to the encoder, wherein the training sample includes an input data sample and a label sample; input the training sample to the first self-attention layer for forward propagation to obtain an actual output corresponding to the input data sample; obtain a loss gradient based on a preset loss function, the actual output and the label sample; and use a back-propagation algorithm to update a weight matrix of the first self-attention layer based on the loss gradient until the weight matrix meets the error requirement.

[0017] Among them, the traditional Transformer model uses a self-attention mechanism to capture the relationship within the sequence and performs a weighted average on all key-value pairs. There is no clear way to distinguish which is key information and which is interference information. For ambiguous query intent, the model may not be able to accurately identify key information because it relies on a single attention mechanism to process all types of semantic information. The present application introduces multiple attention encoders based on a multiple attention mechanism, and each attention encoder learns a different representation of the input sequence to solve the above problem. In order to implement the multiple attention mechanism, the present application obtains the corresponding training samples for each encoder in advance, and uses a preset loss function to train the encoder, thereby obtaining the weight matrix of each encoder. The different weight matrices obtained by training with different training samples can pay attention to different information features during the query statement generation process, further improving the accuracy of query statement generation.

[0018] Optionally, the weight matrix includes self-attention weights, feedforward network weights, layer normalization weights and position encoding weights.

[0019] Among them, each encoder in the present application has an independent weight matrix, including self-attention weights, feedforward network weights, layer normalization and position encoding weights, so that each encoder can focus on different information more accurately.

[0020] Optionally, generating a target query statement according to the context vectors corresponding to the encoders includes: connecting the context vectors corresponding to the encoders to obtain a target context vector; and generating a target query statement according to the target context vector.

[0021] Among them, the present application connects the context vectors obtained by different encoders to obtain a target context vector including all context vectors, and generates a target query statement based on the target context vector. The target context vector includes information obtained by analyzing from multiple angles through different attention heads. Even if there is interference information, some attention heads can identify and focus on key information, allowing the model to analyze and process the query text from different angles. When processing fuzzy queries, different attention heads may capture different clues, helping the model to better understand the query intent and improve the accuracy of query statement generation.

[0022] Optionally, generating a target query statement according to the target context vector includes: decoding the target context vector to obtain a structured query language template; and filling the structured query language template to obtain a target query statement.

[0023] Among them, the present application first generates a structured query language template that accurately reflects the characteristics of the query statement based on the target context vector, and then obtains the target query statement by filling in the template, thereby improving the accuracy and efficiency of query statement generation.

[0024] Optionally, the large language model further includes a query intent identifier, and the query intent identifier is located in a decoder of the large language model;

[0025] The decoding process of the target context vector to obtain a structured query language template includes: inputting the target context vector into the query intent identifier to perform query intent recognition processing through the query intent identifier to obtain at least one key information; and determining a structured query language tag of the structured query language template based on the at least one key information.

[0026] Here, the present application sets up a query intent identifier, which is a module specifically used to identify user query intent. It is crucial for generating accurate SQL templates. In text-to-SQL tasks, users' natural language queries may contain multiple intents. The role of the query intent identifier is to determine the user's query intent, that is, key information, in order to generate correct SQL statements. In the decoder, the query intent identifier can help the model select the correct structure and keywords when generating SQL templates, thereby improving the accuracy of query statement generation.

[0027] Optionally, the target context vector is input into the query intent identifier to perform query intent recognition processing through the query intent identifier to obtain at least one key information, including: inputting the target context vector into the query intent identifier; in the query intent identifier, calculating the target context vector according to a preset weight value, a preset deviation value and a preset activation function in a preset intent recognition function to obtain at least one key information.

[0028] Optionally, the large language model also includes a decoder module and a template-guided attention module, and the decoder module and the template-guided attention module are located in the decoder of the large language model; the decoding processing of the target context vector to obtain a structured query language template includes: inputting the target context vector into the decoder module for decoding processing, and obtaining multiple decoder outputs of the decoder module according to the time step; inputting the multiple decoder outputs into the template-guided attention module; in the template-guided attention module, obtaining a predefined structured query language template; in the template-guided attention module, determining the structure of the structured query language template according to the predefined structured query language template and the multiple decoder outputs.

[0029] Among them, the present application also sets up a template-guided attention module, which is a module specially designed to guide the Transformer model to follow a specific structure when generating SQL templates. This module can help the model generate structured SQL statements during the decoding process, especially when processing fuzzy queries. It can provide guidance when the decoder generates SQL statements, allowing the model to dynamically adjust the generated SQL template according to the input natural language query, which can effectively process fuzzy queries and further improve the accuracy of query statement generation.

[0030] Optionally, in the template-guided attention module, the structure of the structured query language template is determined according to the predefined structured query language template and the multiple decoder outputs, including: in the template-guided attention module, each of the decoder outputs is linearly transformed to obtain multiple linear transformation outputs; in the template-guided attention module, the predefined structured query language template is linearly transformed to obtain a linear transformation template; in the template-guided attention module, each of the linear transformation outputs and the linear transformation template is respectively calculated according to a preset calculation function to obtain multiple attention outputs; in the template-guided attention module, the multiple attention outputs are spliced ​​to obtain the structure of the structured query language template.

[0031] Based on the above method, the current output of the decoder can be aligned with the corresponding part in the predefined SQL template, focusing on guiding the decoder to generate the correct SQL structure.

[0032] Optionally, the decoder module includes a second self-attention layer, a decoding attention layer and a forward propagation module.

[0033] Optionally, the filling processing of the structured query language template to obtain a target query statement includes: identifying placeholders in the structured query language template; obtaining a preset candidate set, wherein the preset candidate set includes multiple replacement items; and filling processing of the placeholders in the structured query language template according to the preset candidate set to obtain the target query statement.

[0034] Optionally, the filling process of the placeholders in the structured query language template according to the preset candidate set to obtain the target query statement includes: using a beam search algorithm to screen the replacement items in the preset candidate set multiple times, and filling the screened replacement items into the placeholders in the structured query language template until all the placeholders are replaced.

[0035] Among them, this application uses the beam search algorithm to fill in the generated structured SQL template to generate a specific SQL query. The Beam Search algorithm can maintain a certain "width" in the search space by retaining the most likely candidate results at each step, while limiting the "depth" of the search to balance the breadth and depth of the search, reduce the amount of calculation, and improve search efficiency. By combining template generation with multiple attention mechanisms and template filling based on BeamSearch, this application can generate diverse SQL queries while maintaining the rationality of the query structure and improving the relevance and accuracy of the generated queries.

[0036] Optionally, obtaining the preset candidate set includes: querying in a preset database to obtain the preset candidate set; and / or obtaining historical query data of the user corresponding to the query text, and generating the preset candidate set based on the historical query data.

[0037] Here, the present application can establish a preset candidate set for filling SQL templates based on a preset database or the user's historical query data corresponding to the query text. It can not only realize accurate SQL template filling through the massive information in the preset database, but also realize customized SQL statement generation based on the user, further improving the flexibility and accuracy of SQL statement generation and improving the user experience.

[0038] Optionally, the vectorizing the query text to obtain a target vector representation corresponding to the query text includes: inputting the query text into a pre-trained language model to obtain a target vector representation corresponding to the query text through an output of the pre-trained language model.

[0039] Among them, the present application can extract the target vector representation corresponding to the query text based on the pre-trained language model, and can efficiently and accurately extract the target vector representation of the query text, providing an accurate and effective basic guarantee for the generation of query statements.

[0040] Optionally, the large language model also includes an input embedding layer and a first linear function layer; the vectorization processing of the query text to obtain a target vector representation corresponding to the query text includes: inputting the query text into the input embedding layer to output the embedding vector corresponding to the query text through the input embedding layer; inputting the embedding vector into the first linear function layer to output the target vector representation corresponding to the query text through the first linear function layer.

[0041] Here, the application converts the input query text into a vector representation in a high-dimensional space through an input embedding layer, and then transforms it into the desired dimension through a linear function layer in the encoder to facilitate subsequent processing.

[0042] In a second aspect, the present application provides a query statement generation device based on a large language model, wherein the large language model includes a plurality of encoders, and the device includes:

[0043] The acquisition module is used to obtain the query text;

[0044] A first processing module, configured to perform vectorization processing on the query text to obtain a target vector representation corresponding to the query text;

[0045] A second processing module is used to input the target vector representation to the multiple encoders to encode the target vector representation through the weight matrix of each encoder to obtain a context vector corresponding to each encoder, wherein the weight matrix of different encoders is different;

[0046] A generation module is used to generate a target query statement according to the context vectors corresponding to each encoder.

[0047] Optionally, the encoder comprises a first self-attention layer and a context vector layer;

[0048] The second processing module is specifically used to: input the target vector representation into the first self-attention layer of the multiple encoders, so as to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain multiple output vectors of each encoder; input the multiple output vectors of each encoder into the corresponding context vector layer, so as to perform weighted processing on the multiple output vectors through each context vector layer to obtain the context vector corresponding to each encoder.

[0049] Optionally, before the second processing module is used to input the target vector representation to the first self-attention layer of the multiple encoders so as to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain multiple output vectors of each encoder, the above-mentioned device also includes a training module, which is used to: for each encoder, obtain a training sample corresponding to the encoder, wherein the training sample includes an input data sample and a label sample; input the training sample to the first self-attention layer for forward propagation to obtain an actual output corresponding to the input data sample; obtain a loss gradient based on a preset loss function, the actual output and the label sample; and use a back-propagation algorithm to update the weight matrix of the first self-attention layer according to the loss gradient until the weight matrix meets the error requirement.

[0050] Optionally, the weight matrix includes self-attention weights, feedforward network weights, layer normalization weights and position encoding weights.

[0051] Optionally, the generating module includes:

[0052] A concatenation submodule, configured to perform concatenation processing on the context vectors corresponding to the encoders to obtain a target context vector;

[0053] The generating submodule is used to generate a target query statement according to the target context vector.

[0054] Optionally, the generating submodule further includes:

[0055] A decoding submodule, used for decoding the target context vector to obtain a structured query language template;

[0056] The filling submodule is used to fill the structured query language template to obtain a target query statement.

[0057] Optionally, the large language model also includes a query intent identifier, which is located in the decoder of the large language model; the decoding submodule is specifically used to: input the target context vector into the query intent identifier to perform query intent recognition processing through the query intent identifier to obtain at least one key information; and determine the structured query language tag of the structured query language template based on the at least one key information.

[0058] Optionally, the decoding submodule is also specifically used to: input the target context vector into the query intent identifier; in the query intent identifier, calculate the target context vector according to a preset weight value, a preset bias value and a preset activation function in a preset intent recognition function to obtain at least one key information.

[0059] Optionally, the large language model further includes a decoder module and a template-guided attention module, wherein the decoder module and the template-guided attention module are located in the decoder of the large language model; the decoding submodule is specifically used for:

[0060] The target context vector is input into the decoder module for decoding, and a plurality of decoder outputs of the decoder module are obtained according to the time step; the plurality of decoder outputs are input into the template-guided attention module; in the template-guided attention module, a predefined structured query language template is obtained; in the template-guided attention module, a structure of the structured query language template is determined according to the predefined structured query language template and the plurality of decoder outputs.

[0061] Optionally, the decoding submodule is also specifically used to: in the template-guided attention module, perform linear transformation processing on each of the decoder outputs to obtain multiple linear transformation outputs; in the template-guided attention module, perform linear transformation processing on the predefined structured query language template to obtain a linear transformation template; in the template-guided attention module, perform calculation processing on each of the linear transformation outputs and the linear transformation template according to a preset calculation function to obtain multiple attention outputs; in the template-guided attention module, perform splicing processing on the multiple attention outputs to obtain the structure of the structured query language template.

[0062] Optionally, the decoder module includes a second self-attention layer, a decoding attention layer and a forward propagation module.

[0063] Optionally, the filling submodule is specifically used to: identify placeholders in the structured query language template; obtain a preset candidate set, wherein the preset candidate set includes multiple replacement items; and fill the placeholders in the structured query language template according to the preset candidate set to obtain a target query statement.

[0064] Optionally, the filling submodule is further specifically configured to: adopt a beam search algorithm to screen the replacement items in the preset candidate set multiple times, and fill the screened replacement items into the placeholders in the structured query language template until all the placeholders are replaced.

[0065] Optionally, the acquisition module is specifically used to: perform a query in a preset database to obtain a preset candidate set; and / or, obtain historical query data of the user corresponding to the query text, and generate a preset candidate set based on the historical query data.

[0066] Optionally, the first processing module is specifically used to: input the query text into a pre-trained language model to obtain a target vector representation corresponding to the query text through an output of the pre-trained language model.

[0067] Optionally, the large language model also includes an input embedding layer and a first linear function layer; the first processing module is also specifically used to: input the query text into the input embedding layer to output the embedding vector corresponding to the query text through the input embedding layer; input the embedding vector into the first linear function layer to output the target vector representation corresponding to the query text through the first linear function layer.

[0068] In a third aspect, the present application provides a query statement generation device based on a large language model, comprising: at least one processor and a memory;

[0069] The memory stores computer-executable instructions;

[0070] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the query statement generation method based on the large language model as described in the first aspect and various possible designs of the first aspect.

[0071] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the query statement generation method based on the large language model as described in the first aspect and various possible designs of the first aspect is implemented.

[0072] In a fifth aspect, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the query statement generation method based on a large language model as described in the first aspect and various possible designs of the first aspect.

[0073] The present application provides a query statement generation method, device, equipment and medium based on a large language model, wherein the method uses a large language model to generate query statements, and the large language model is improved on the basis of a general Transformer large model, and multiple encoders are set. For the corresponding target vector representation of the query text, the target vector representation is encoded based on the weight matrix in each encoder through multiple encoders of the large language model, and each encoder outputs a context vector, and then generates a target query statement based on the context vectors output by different encoders. Since the weight matrix in each encoder is different, each encoder can pay attention to the different semantic and structural features of the query text when extracting features, so that the model can generate specific templates in different subspaces, allowing the model to focus on different information features in different attention heads, independently capture different aspects of the query text, and process information at multiple abstract levels, so as to better capture complex and abstract semantic relationships, and can distinguish the importance of information well when dealing with complex and ambiguous questions, thereby improving the accuracy of query statement generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0075] Figure 1 A schematic diagram of the structure of a query statement generation system based on a large language model provided in an embodiment of the present application;

[0076] Figure 2 A schematic diagram of the structure of a large language model provided in an embodiment of the present application;

[0077] Figure 3 A flowchart of a query statement generation method based on a large language model provided in an embodiment of the present application;

[0078] Figure 4 A flowchart of another method for generating a query statement based on a large language model provided in an embodiment of the present application;

[0079] Figure 5 A flowchart of another method for generating a query statement based on a large language model provided in an embodiment of the present application;

[0080] Figure 6A flowchart of another method for generating a query statement based on a large language model provided in an embodiment of the present application;

[0081] Figure 7 A schematic diagram of the structure of another large language model provided in an embodiment of the present application;

[0082] Figure 8 A schematic diagram of the structure of a query statement generation device based on a large language model provided in an embodiment of the present application;

[0083] Fig. 9 A schematic diagram of the structure of a query statement generation device based on a large language model provided in an embodiment of the present application.

[0084] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0085] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0087] In recent years, with the rapid development of large language models (LLM), new solutions have been provided for text-to-SQL tasks. Large language models are natural language processing models trained by machine learning using large-scale corpora. Large language models can understand the complex semantics and contextual information in natural language queries, which enables them to be more accurately converted into corresponding SQL queries. The general Transformer large model is a widely used large language model. However, the current text-to-SQL technology based on large language models still faces the following challenges: No support for fuzzy queries: The questions raised by existing text-to-SQL users may not be expressed clearly or specifically enough. At this time, the intention recognition of fuzzy questions is particularly important. By accurately understanding the user's query intent, the model can generate SQL queries that better meet the user's needs. The method often has difficulty processing queries containing fuzzy matches, such as partial matches of keywords. In practical applications, users may raise very complex queries containing multiple conditions and subqueries. In such cases, the ability to identify intent and filter irrelevant information is particularly important. Unrestricted interference from irrelevant text cannot be ruled out: In practical applications, user queries may contain a large amount of irrelevant information, which will affect the accuracy of the query results. When asking questions, users may include some words or details that are irrelevant to database queries. If the model cannot effectively identify and filter this information, the generated SQL statement may contain incorrect conditions or structures, thereby returning incorrect results. Based on the above, it can be seen that the existing technology has the technical problem of low accuracy in query statement generation.

[0088] In order to solve the above problems, the embodiments of the present application provide a query statement generation method, device, equipment and medium based on a large language model. The method uses a large language model based on a multiple attention mechanism to generate query statements. The large language model is improved on the basis of the general Transformer large model, and multiple encoders are set. For the corresponding target vector representation of the query text, the target vector representation is encoded based on the weight matrix in each encoder through multiple encoders of the large language model, and each encoder outputs a context vector. Then, based on the context vectors output by different encoders, a target query statement is generated.

[0089] In one possible implementation, the general Transformer large model needs to be specially modified to better handle the text to SQL task. Although the general model has powerful language understanding capabilities, it is not optimized for the specific semantics and structured output of SQL queries. The necessity of the modification is reflected in the need for the model to accurately capture the intent of the user's query, understand the complex associations of the database schema, and ensure the grammatical and logical correctness of the generated SQL statements. The text to SQL task places higher requirements on the model's decoding efficiency and long sequence processing capabilities. Therefore, through customized attention mechanisms, adjustments to the input and output layers, and training strategies tailored to SQL characteristics, the model's performance on the text to SQL task can be significantly improved, achieving more efficient and accurate query statement generation. The general Transformer model uses a self-attention mechanism to simultaneously focus on the positions of all input sequences, but this mechanism may not be able to distinguish the importance of information well when dealing with complex and ambiguous questions. The embodiment of the present application proposes a multiple attention mechanism, which allows the model to focus on different information features in different attention heads. (1) Some attention heads can be used specifically to identify and focus on keywords, while other attention heads can be used to understand complex relationships or ambiguous expressions in context. This fine-grained focus helps the model identify and prioritize key information while filtering out irrelevant content. (2) The multi-attention mechanism implements multiple attention modes through different attention heads, wide attention (broadly capturing context) and narrow attention (focusing on key information). This diversity helps the model better understand complex and ambiguous questions because it can analyze the input text from multiple perspectives. Certain attention heads can be trained to specifically identify and interpret ambiguous expressions, such as metaphors, ellipsis, or ambiguous statements. (3) The multi-attention mechanism can focus on different semantic or structural features through different attention heads. Even in the presence of interference information, certain attention heads are able to identify and focus on key information. For example, one attention head may focus on entity recognition while another focuses on semantic role labeling. The multi-attention mechanism allows the model to examine the input sequence from different perspectives. When processing ambiguous queries, different attention heads may capture different clues to help the model better understand the query intent.

[0090] Optional, Figure 1 A schematic diagram of a query statement generation system architecture based on a large language model provided in an embodiment of the present application. Figure 1 In the above architecture, the above architecture includes at least one of a data acquisition device 101, a processing device 102 and a display device 103.

[0091] It is understandable that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the architecture of the query statement generation system based on the large language model. In other feasible implementations of the present application, the above architecture may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0092] In a specific implementation process, the data acquisition device 101 may include an input / output interface and may also include a communication interface. The data acquisition device 101 may be connected to the processing device via the input / output interface or the communication interface.

[0093] The processing device 102 can encode the target vector representation corresponding to the query text through multiple encoders of the large language model based on the weight matrix in each encoder, output a context vector through each encoder, and then generate a target query statement based on the context vectors output by different encoders.

[0094] The display device 103 may also be a touch display screen or a screen of a terminal device, which is used to receive user instructions while displaying the above content to achieve interaction with the user.

[0095] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions in a memory and executing the instructions, or it can be implemented by a chip circuit.

[0096] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0097] The technical solution of the present application is described in detail below in conjunction with specific embodiments:

[0098] Optionally, Figure 2 A schematic diagram of the structure of a large language model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the large language model provided in the embodiment of the present application includes a first encoder 21, a second encoder 22 and a third encoder 23, and each encoder includes a weight matrix different from that of other encoders.

[0099] It is understandable that the large language model can also include other structures. Figure 1 A specific display is given in.

[0100] It can be understood that in a large language model, the number of encoders can be determined according to actual conditions, and the number of encoders is greater than 1 to implement multiple attention mechanisms through different encoding matrices in multiple encoders.

[0101] Optionally, based on the large language model in the embodiment of the present application, it is possible to use multiple encoders of the large language model to encode the target vector representation based on the weight matrix in each encoder, output a context vector through each encoder, and then generate a target query statement based on the context vectors output by different encoders.

[0102] Optionally, Figure 3 A flowchart of a query statement generation method based on a large language model provided in an embodiment of the present application. The execution subject of the embodiment of the present application can be Figure 1 The query statement generation system based on the large language model in the , the specific execution subject can be determined according to the actual application scenario. Figure 3 As shown, the method comprises the following steps:

[0103] S301: Obtain query text.

[0104] The query text here may be a query question input by a user, for example: list the names and email addresses of all users who have purchased product X.

[0105] S302: Perform vectorization processing on the query text to obtain a target vector representation corresponding to the query text.

[0106] Optionally, the query text is vectorized to obtain a target vector representation corresponding to the query text, including: inputting the query text into a pre-trained language model to obtain a target vector representation corresponding to the query text through the output of the pre-trained language model.

[0107] Optionally, the pre-trained language model is a Bidirectional Encoder Representations from Transformers (BERT) model. Use a pre-trained natural language processing model (such as BERT) to encode the question and obtain a vector representation that captures the semantic information of the question. This vector representation will serve as the basis for the multi-attention mechanism.

[0108] Among them, the embodiment of the present application can extract the target vector representation corresponding to the query text based on the pre-trained language model, and can efficiently and accurately extract the target vector representation of the query text, providing an accurate and effective basic guarantee for the generation of query statements.

[0109] Optionally, the large language model also includes an input embedding layer and a first linear function layer; vectorizing the query text to obtain a target vector representation corresponding to the query text, including: inputting the query text into the input embedding layer to output the embedding vector corresponding to the query text through the input embedding layer; inputting the embedding vector into the first linear function layer to output the target vector representation corresponding to the query text through the first linear function layer.

[0110] Among them, the input embedding layer and the linear function layer are placed before the encoder.

[0111] Input Embedding Layer: Since machine learning models cannot directly understand text data, the task of the input embedding layer is to convert the input words or characters into vector representations in a high-dimensional space, namely embeddings. Each word or character is assigned a unique vector that captures the semantic and syntactic information of the word or character. Positional encoding is usually added to these embedding vectors to preserve the position information of the word in the original sequence, because the embedding itself does not contain position information.

[0112] Linear Function Layers: Linear functions are mainly present before each sub-layer in this model to transform the input into the desired dimension. These linear transformations are usually implemented by multiplying the weight matrix by the input vector plus the bias vector. They are used in both encoder and decoder layers. Forward propagation refers to the process of input data passing through the entire model. Within each layer, the data passes through multiple sub-layers, including self-attention layers and feed-forward neural networks.

[0113] Here, the embodiment of the present application converts the input query text into a vector representation in a high-dimensional space through an input embedding layer, and then transforms it into the desired dimension through a linear function layer in the encoder to facilitate subsequent processing.

[0114] S303: Input the target vector representation to multiple encoders to encode the target vector representation through the weight matrix of each encoder to obtain a context vector corresponding to each encoder.

[0115] Among them, different encoders have different weight matrices.

[0116] Among them, the embodiment of the present application adopts a multiple attention mechanism, the main principles of which are as follows:

[0117] The original Transformer model uses a self-attention mechanism to capture the relationship within the sequence. Self-attention is calculated by the similarity between (Query) and key (Key). For example, the dot product multi-attention mechanism can be used, and its formula is (1):

[0118] Attention(Q,K,V)=softmax(qK T )V Formula (1)

[0119] In formula (1), Attention(Q,K,V) represents the attention solution result, Q is the query, K is the key, V is the value, and K is the T is the dimension of the key. In this example, the query can be the current state of the decoder, and the key and value can be vector representations of the question encoding. The attention mechanism of the original Transformer may not be robust enough when dealing with queries that contain a lot of interference information. This is because self-attention performs a weighted average of all key-value pairs, and there is no clear way to distinguish which is key information and which is interference information. For ambiguous query intent, the model may not be able to accurately identify key information because it relies on a single attention mechanism to process all types of semantic information. It is understandable that in any formula in this application, the specific meaning of its parameters and parameter representations only represents the explanation in this formula, and is not applicable to other formulas. For the specific meaning of the parameters in each formula, please refer to the explanation of each formula.

[0120] The multi-attention mechanism can solve the above problem by introducing multiple attention encoders, each of which learns a different representation of the input sequence, as shown in the following formula (2).

[0121] MultiAttention(Q,K,V)=Concat(Encoder1,...,Encoder h )W formula (2)

[0122] In formula (2), MultiAttention(Q,K,V) represents the attention solution, h is the number of attention encoders, Q is the query, K is the key, V is the value, and W is the weight matrix of each encoder. Each attention encoder has an independent weight matrix Different weight matrices enable each attention encoder to learn in different subspaces, thereby improving the model's ability to identify key information.

[0123] S304: Generate a target query statement according to the context vectors corresponding to each encoder.

[0124] The present application proposes a query statement generation method based on a large language model, which uses a large language model based on a multiple attention mechanism to generate query statements. The large language model is improved on the basis of the general Transformer large model, and multiple encoders are set. For the corresponding target vector representation of the query text, the target vector representation is encoded based on the weight matrix in each encoder through multiple encoders of the large language model, and the context vector is output by each encoder. Then, based on the context vectors output by different encoders, the target query statement is generated. Since the weight matrix in each encoder is different, each encoder can pay attention to the different semantic and structural features of the query text when extracting features, so that the model can generate specific templates in different subspaces, allowing the model to focus on different information features in different attention heads, independently capture different aspects of the query text, and process information at multiple abstract levels, so as to better capture complex and abstract semantic relationships, and can distinguish the importance of information well when dealing with complex and ambiguous questions, thereby improving the accuracy of query statement generation.

[0125] Optionally, the encoder includes a first self-attention layer and a context vector layer, and accordingly, Figure 4 A flowchart of another method for generating a query statement based on a large language model provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the method includes:

[0126] S401: Obtain query text.

[0127] S402: Perform vectorization processing on the query text to obtain a target vector representation corresponding to the query text.

[0128] The implementation of steps S401 and S402 is the same as that of steps S301 and S302, and will not be described in detail here.

[0129] S403: Input the target vector representation to the first self-attention layer of multiple encoders to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain multiple output vectors of each encoder.

[0130] S404: Input multiple output vectors of each encoder into the corresponding context vector layer, so as to perform weighted processing on the multiple output vectors through each context vector layer to obtain the context vector corresponding to each encoder.

[0131] The weight matrices for different encoders are different. The self-attention layer allows the model to assign different weights to each word in the input sequence, based on the relationship between the word and all other words. Self-attention works by calculating three vectors: query, key, and value. Each input word generates these three vectors, then the dot product between all queries and keys is calculated to get the attention weights, and finally these weights are used to weight the value vector to generate the output vector. Scaling dot product attention is a technique to prevent the inner product from being too large, causing the gradient to disappear or explode.

[0132] Context vector layer: On the decoder side, each decoder contains a context vector (also called the output of "encoder-decoder attention"), which represents a weighted combination of the encoder output sequence. This context vector is used to generate the next decoder output. The context vector is a special self-attention mechanism in the decoder, which helps the decoder focus on the relevant parts of the encoder output.

[0133] Optionally, before inputting the target vector representation to the first self-attention layers of the plurality of encoders to calculate the target vector representation by the weight matrix of the first self-attention layer of each encoder to obtain the plurality of output vectors of each encoder, the method further includes:

[0134] For each encoder, obtain the training samples corresponding to the encoder, wherein the training samples include input data samples and label samples; input the training samples to the first self-attention layer for forward propagation to obtain the actual output corresponding to the input data sample; obtain the loss gradient according to the preset loss function, the actual output and the label sample; use the back-propagation algorithm to update the weight matrix of the first self-attention layer according to the loss gradient until the weight matrix meets the error requirements.

[0135] Among them, each encoder focuses on different points, so the training samples corresponding to each encoder are different.

[0136] Among them, the traditional Transformer model uses a self-attention mechanism to capture the relationship within the sequence and performs a weighted average on all key-value pairs. There is no clear way to distinguish which is key information and which is interference information. For ambiguous query intent, the model may not be able to accurately identify key information because it relies on a single attention mechanism to process all types of semantic information. The embodiment of the present application introduces multiple attention encoders based on a multiple attention mechanism, and each attention encoder learns a different representation of the input sequence to solve the above problem. In order to realize the multiple attention mechanism, the embodiment of the present application obtains the corresponding training samples for each encoder in advance, and uses a preset loss function to train the encoder, thereby obtaining a weight matrix for each encoder. The different weight matrices obtained by training with different training samples can pay attention to different information features during the query statement generation process, further improving the accuracy of query statement generation.

[0137] Optionally, the weight matrix includes self-attention weights, feedforward network weights, layer normalization weights, and position encoding weights.

[0138] Among them, each encoder in the embodiment of the present application has an independent weight matrix, including self-attention weights, feedforward network weights, layer normalization and position encoding weights, so that each encoder can focus on different information more accurately.

[0139] S405: Generate a target query statement according to the context vectors corresponding to each encoder.

[0140] Here, the encoder of the embodiment of the present application includes a self-attention layer for assigning different attention weights to the target vector representation so as to extract semantic information in a targeted manner, and a context vector layer for weighted processing to obtain a context vector. The self-attention layer is used to focus on different semantic features and flexibly and accurately identify multiple key information in the query text. The context vector layer can output a context vector to help the large model focus on the relevant parts of the encoder output, further improving the accuracy of query statement generation.

[0141] Optionally, the embodiment of the present application can accurately generate a target query sentence based on the output of multiple encoders. Accordingly, Figure 5 A flowchart of another method for generating a query statement based on a large language model provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the method includes:

[0142] S501: Obtain query text.

[0143] S502: Perform vectorization processing on the query text to obtain a target vector representation corresponding to the query text.

[0144] S503: Input the target vector representation to multiple encoders to encode the target vector representation through the weight matrix of each encoder to obtain a context vector corresponding to each encoder.

[0145] Among them, different encoders have different weight matrices.

[0146] The implementation of steps S501-S503 is the same as that of steps S401-S404, and will not be described in detail here.

[0147] S504: Connect the context vectors corresponding to the encoders to obtain a target context vector.

[0148] S505: Generate a target query statement according to the target context vector.

[0149] Optionally, generating a target query statement according to the target context vector includes: decoding the target context vector to obtain a structured query language template; and filling the structured query language template to obtain the target query statement.

[0150] Among them, the embodiment of the present application first generates a structured query language template that accurately reflects the characteristics of the query statement based on the target context vector, and then obtains the target query statement by filling in the template, thereby improving the accuracy and efficiency of query statement generation.

[0151] The embodiment of the present application connects the context vectors obtained by different encoders to obtain a target context vector including all context vectors, and generates a target query statement based on the target context vector. The target context vector includes information obtained by analyzing from multiple angles by different attention heads. Even if there is interference information, some attention heads can identify and focus on key information, allowing the model to analyze and process the query text from different angles. When processing fuzzy queries, different attention heads may capture different clues, helping the model to better understand the query intent and improve the accuracy of query statement generation.

[0152] Optionally, the large language model also includes a query intent identifier, which is located in the decoder of the large language model; decoding the target context vector to obtain a structured query language template, including: inputting the target context vector into the query intent identifier to perform query intent recognition processing through the query intent identifier to obtain at least one key information; determining a structured query language tag of the structured query language template based on the at least one key information.

[0153] Here, the embodiment of the present application sets up a query intent identifier, which is a module specifically used to identify user query intent. It is crucial for generating accurate SQL templates. In text-to-SQL tasks, users' natural language queries may contain multiple intents. The role of the query intent identifier is to determine the user's query intent, that is, key information, in order to generate correct SQL statements. In the decoder, the query intent identifier can help the model select the correct structure and keywords when generating SQL templates, thereby improving the accuracy of query statement generation.

[0154] Optionally, the target context vector is input into a query intent identifier to perform query intent recognition processing through the query intent identifier to obtain at least one key information, including: inputting the target context vector into the query intent identifier; in the query intent identifier, calculating the target context vector according to a preset weight value, a preset deviation value and a preset activation function in a preset intent recognition function to obtain at least one key information.

[0155] Query intent identifier: This is a module dedicated to identifying the user's query intent, which is crucial for generating accurate SQL templates. In text-to-SQL tasks, the user's natural language query may contain multiple intents, such as "query" (SELECT), "update" (UPDATE), "insert" (INSERT), or "delete" (DELETE). The role of the query intent identifier is to determine the user's query intent in order to generate the correct SQL statement. In the decoder, the query intent identifier can help the model select the correct structure and keywords when generating SQL templates.

[0156] In a possible implementation, the query intent identifier works as follows: Assume that there is a decoder input sequence Z, which contains the context information output by the encoder. The goal of the query intent identifier is to predict the user's query intent y, which can be one of multiple intent categories. The query intent identifier can be expressed as a function, as shown in formula (3):

[0157] y=f(Z)=W T Relu(WZ+b) Formula (3)

[0158] In formula (3), W and b are the weight matrix and bias of the query intent identifier, ReLU is the activation function, and W and b can be determined according to the actual situation. TRepresents the dimension of the weight matrix W. The weight matrix W of the query intent identifier can be learned through training so that the model can predict the correct query intent based on the features of the input sequence Z. During the decoding process, the output y of the query intent identifier can be used to guide the decoder to generate the correct SQL template. For example, if the identifier predicts the query intent as "query", the decoder will start generating a SQL template starting with "SELECT".

[0159] Optionally, the large language model also includes a decoder module and a template-guided attention module, and the decoder module and the template-guided attention module are located in the decoder of the large language model; the target context vector is decoded to obtain a structured query language template, including: inputting the target context vector into the decoder module for decoding, and obtaining multiple decoder outputs of the decoder module according to the time step; inputting the multiple decoder outputs into the template-guided attention module; in the template-guided attention module, obtaining a predefined structured query language template; in the template-guided attention module, determining the structure of the structured query language template according to the predefined structured query language template and the multiple decoder outputs.

[0160] Among them, the embodiment of the present application also sets up a template-guided attention module, which is a module specially designed to guide the Transformer model to follow a specific structure when generating SQL templates. This module can help the model generate structured SQL statements during the decoding process, especially when processing fuzzy queries. It can provide guidance when the decoder generates SQL statements, allowing the model to dynamically adjust the generated SQL template according to the input natural language query, which can effectively process fuzzy queries and further improve the accuracy of query statement generation.

[0161] Optionally, in the template-guided attention module, the structure of the structured query language template is determined according to a predefined structured query language template and multiple decoder outputs, including: in the template-guided attention module, each decoder output is linearly transformed to obtain multiple linearly transformed outputs; in the template-guided attention module, a predefined structured query language template is linearly transformed to obtain a linearly transformed template; in the template-guided attention module, each linear transformation output and linear transformation template are respectively calculated according to a preset calculation function to obtain multiple attention outputs; in the template-guided attention module, multiple attention outputs are spliced ​​to obtain the structure of the structured query language template.

[0162] In one possible implementation, the template-guided attention module works as follows: The template-guided attention module can be designed as an attention mechanism that aligns the current output of the decoder with the corresponding part in the predefined SQL template. This module can be regarded as an additional attention head that focuses on guiding the decoder to generate the correct SQL structure. Suppose we have a decoder output h at time t t , and a predefined SQL template T = (t1, t2, ..., t n ). The calculation of the template-guided attention module can be expressed as the following formula (4):

[0163] TemplateAttention(h t ,T)=tan(QK)·V Formula (4)

[0164] Where, in formula (4), Q is the decoder output h t The output of the attention mechanism can be regarded as the template-guided representation of the decoder at time t. This representation can be combined with the output of other attention heads of the decoder or the output h of the previous time step t-1 Concatenate to generate the final decoder output y t :

[0165] y t =Decoder(TemplateAttention(h t ,T);h t-1 ; ...)

[0166] In this way, the template-guided attention module can dynamically provide guidance from SQL templates during the decoding process, helping the model generate structured SQL statements. This mechanism is particularly useful for handling ambiguous queries because it provides a structured framework based on which the decoder can fill in specific query details.

[0167] Based on the above method, the current output of the decoder can be aligned with the corresponding part in the predefined SQL template, focusing on guiding the decoder to generate the correct SQL structure.

[0168] Optionally, the decoder module includes a second self-attention layer, a decoding attention layer, and a forward propagation module.

[0169] Optionally, on the decoder side, each decoder contains a context vector (also called the output of "encoder-decoder attention"), which represents a weighted combination of the encoder output sequence. This context vector is used to generate the next decoder output. The context vector is a special self-attention mechanism in the decoder that helps the decoder focus on the relevant parts of the encoder output.

[0170] Optionally, decoder attention is a part of the decoder that combines self-attention with context vectors. With self-attention, we focus on previous tokens in the sequence currently being decoded (preventing future information leakage). With encoder-decoder attention, we focus on the context vector output by the encoder, ensuring that the decoder output is consistent with the context of the input sequence.

[0171] In step S503, the embodiment of the present application introduces a multi-attention layer in the encoder-decoder framework. When generating each part of the SQL template, the decoder focuses multi-attention on the key information of the question encoding. The multi-attention mechanism associates the question encoding with the current state of the decoder by calculating the multi-attention weights. The multi-attention weights reflect the importance of each word in the question to the current decoder state. For example, keywords such as "list", "name", and "email" have special weights. Based on the multi-attention mechanism, an optional method of step S505 is described below: Based on the multi-attention mechanism, the specific method of generating a query statement is as follows:

[0172] Step 505.1. Calculate multiple attention weights for the target context vector: Based on the multiple attention weights, the decoder generates a specific SQL template. The multiple attention weights determine which question words will have a greater impact on the current decoder state. For example, when the decoder needs to generate a SELECT clause, "list" in the question may get a higher attention weight. When the decoder needs to generate a WHERE clause, "product X" in the question may get a higher attention weight. This process involves how multiple encoders work together to identify key information in user queries and map it to different parts of the SQL template. In this way, the model is able to handle fuzzy queries and generate accurate SQL templates. This part can be performed by the query intent identifier.

[0173] SELECT: Attention may identify trigger words such as "select" or "query" and give them high attention weights. An example of generating a SQL template based on the trigger words is as follows:

[0174]

[0175] Table name (FROM): A focus may be focused on identifying the table name.

[0176]

[0177] Columns (WHERE): One focus might be on identifying the table name.

[0178]

[0179] Condition (=): One attention may be focused on identifying the table name.

[0180]

[0181] Value: A focus may be focused on identifying the table name.

[0182]

[0183] By combining the outputs of these attention heads, the decoder can gradually build the SQL template

[0184] Step 505.2, generate the key part of the SQL template: The decoder generates the key part of the SQL template based on the context vector generated by the multi-attention mechanism. This part consists of, for example, when generating a SELECT clause, the decoder pays attention to the "user name and email address" in the question and maps this information to the column names in the SQL template. The multi-attention weights determine which question words will have a greater impact on the current decoder state, thereby affecting the key part of the generated SQL template. The current state of the decoder and the context vector calculated by the multi-attention mechanism are used to predict the next SQL token. This part can be performed by the template-guided attention module, which is specifically expressed as follows:

[0185] SQL_template

[0186] =Decode(select_head,from_head,where_head,condition_head,value_head)

[0187] The decoder will output a series of SQL tags, including SELECT, FROM, WHERE, etc.

[0188] The generated SQL template is as follows:

[0189] SELECT <column1> , <column2>FROM <column3>='value'

[0190] Optionally, after generating the SQL template, you need to use the template to fill in and generate various SQL queries. Figure 6 A flowchart of another method for generating a query statement based on a large language model provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the method includes:

[0191] S601: Obtain query text.

[0192] S602: Perform vectorization processing on the query text to obtain a target vector representation corresponding to the query text.

[0193] S603: Input the target vector representation to multiple encoders to encode the target vector representation through the weight matrix of each encoder to obtain a context vector corresponding to each encoder.

[0194] Among them, different encoders have different weight matrices.

[0195] S604: Connect the context vectors corresponding to the encoders to obtain a target context vector.

[0196] The implementation of steps S601-S604 is the same as that of steps S501-S504, and will not be described in detail here.

[0197] S605: Decode the target context vector to obtain a structured query language template.

[0198] The implementation of step S605 may refer to the content of the above embodiment S505, which will not be elaborated here.

[0199] S606: Fill in the structured query language template to obtain a target query statement.

[0200] Optionally, the structured query language template is filled in to obtain a target query statement, including: identifying placeholders in the structured query language template; obtaining a preset candidate set, wherein the preset candidate set includes multiple replacement items; and filling in the placeholders in the structured query language template according to the preset candidate set to obtain the target query statement.

[0201] Optionally, according to a preset candidate set, the placeholders in the structured query language template are filled to obtain a target query statement, including: using a beam search algorithm to screen the replacement items in the preset candidate set multiple times, and filling the screened replacement items into the placeholders in the structured query language template until all placeholders are replaced.

[0202] Among them, the embodiment of the present application uses the Beam Search algorithm to fill in the generated structured SQL template to generate a specific SQL query. The Beam Search algorithm can maintain a certain "width" in the search space by retaining the most likely candidate results at each step, while limiting the "depth" of the search to balance the breadth and depth of the search, reduce the amount of calculation, and improve the search efficiency. By combining template generation with multiple attention mechanisms and template filling based on BeamSearch, the embodiment of the present application can generate diversified SQL queries while maintaining the rationality of the query structure and improving the relevance and accuracy of the generated queries.

[0203] Optionally, obtaining a preset candidate set includes: performing a query in a preset database to obtain the preset candidate set; and / or obtaining historical query data of the user corresponding to the query text, and generating the preset candidate set based on the historical query data.

[0204] Here, the embodiment of the present application can establish a preset candidate set for filling the SQL template based on the preset database or the user's historical query data corresponding to the query text. It can not only realize accurate SQL template filling through the massive information in the preset database, but also realize customized SQL statement generation based on the user, further improving the flexibility and accuracy of SQL statement generation and improving the user experience.

[0205] Based on the generated structured SQL template, the Beam Search algorithm is used to fill it in to generate a specific SQL query. The Beam Search algorithm can maintain a certain "width" in the search space by retaining the most likely candidate results at each step, while limiting the "depth" of the search to balance the breadth and depth of the search, reduce the amount of calculation, and improve the search efficiency. By combining template generation with multiple attention mechanisms and template filling based on Beam Search, the present invention can generate diversified SQL queries while maintaining the legitimacy of the query structure and improving the relevance and accuracy of the generated queries.

[0206] In a possible implementation, the steps of using template filling to generate a variety of SQL queries are as follows:

[0207] S606.1. Initialize Beam Search: Use the Beam Search algorithm to initialize the query generation process. At each step, Beam Search retains a pre-set number of candidate queries (e.g., the first 5). Beam Search is a heuristic search algorithm that is mainly used for sequence generation tasks in fields such as natural language processing (NLP) and machine translation, such as automatically generating text or translating sentences. The core idea is to retain the most likely candidate results at each step instead of retaining only one optimal result, thereby maintaining a certain "width" in the search space while limiting the "depth" of the search to balance the breadth and depth of the search, reduce the amount of calculation, and improve the search efficiency. Beam search searches on the probability distribution of the output of the generative model or decoder: given a starting symbol (such as the start tag of a sentence or the beginning of an SQL query), the model predicts the probability distribution of the first element. Beam Search selects the elements with the highest probability (e.g., the first 5), which are called "beams". For each candidate element, the model predicts the probability distribution of the next element. Then Beam Search selects the next element with the highest probability for each candidate element and adds these new elements to the corresponding candidate sequence. Based on the probability of the new candidate sequence, Beam Search updates the beam and retains the sequence with the highest probability. This process is repeated until the end condition of the sequence is reached (such as the end of the sentence marker or the end of the SQL query). Beam Search outputs the sequences with the highest probability as the final result.

[0208] S606.2. Select and fill the first placeholder:

[0209] The first placeholder in the template is WHERE . Based on the database schema, there is a whitelist of table names, for example: "users", "orders", "products". Beam Search populates

[0210]

[0211] i

[0212]

[0213] <column1> , <column2>FROM users WHERE <column3>='value'

[0214] S606.4. Select and fill the second placeholder:

[0215] Now need to fill in" <column1>"and" <column2>". Based on the schema of the "users" table, we have a whitelist of column names, such as: "id", "name", "email". Beam Search selects " <column1>"and" <column2>", all possible column names are considered and the column name with the highest probability is chosen as the fill. For example, "id" and "name" might be the column names with the highest probability.

[0216] S606.5. Fill in the second placeholder:

[0217] Assuming "id" and "name" are the selected column names, the query template now looks like this:

[0218] SELECT id,name FROM users WHERE <column3>='value'

[0219] S606.6. Select and fill in the third placeholder:

[0220] Now need to fill in" <column3>". According to the schema of the "users" table, continue to use the whitelist of column names. BeamSearch selects " <column3>", all possible column names are considered and the column name with the highest probability is chosen as the fill. For example, "email" might be the column name with the highest probability.

[0221] S606.7. Fill in the third placeholder: Assuming "email" is the selected column name, the final query template now looks like this:

[0222] SELECT id,name FROM users WHERE email='value'

[0223] Through the above steps, a specific SQL query is generated using the template filling. Throughout the process, BeamSearch maintains multiple candidate queries and selects the filling option with the highest probability at each step. This approach ensures that the generated query not only conforms to the template structure but is also based on the actual database schema, thereby improving the relevance and accuracy of the query.

[0224] Optionally, Figure 7 A schematic diagram of the structure of another large language model provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the large language model provided by the embodiment of the present application includes two encoders, each of which includes two encoding layers. A linear function layer and two encoding layers constitute an encoder. The multiple attention mechanism is implemented by two encoders, and the functions of eliminating interference text and supporting fuzzy queries can be realized in the text-to-sql task. Among them, each encoding layer adopts a forward propagation mechanism and a self-attention mechanism. It also includes a decoder, which may include two decoding layers, a linear function layer and a query intent identifier, and each decoding layer adopts template guidance, forward propagation, decoding attention and self-attention mechanisms. In order to further solve the model query and exclude irrelevant statements during inquiry, a query intent identifier and a template-guided attention module are also provided. Both the query intent identifier and the template-guided attention module can be set in the decoder.

[0225] It should be noted that Figure 7 The structure of each encoder is the same. For example, the left encoder includes an encoding layer 1 and an encoding layer 2, and the right encoder includes an encoding layer 1 and an encoding layer 2. The same encoder 1 only refers to the two encoding layers with the same structure and function. Both encoders include an input embedding layer, and the functions and structures of the two input embedding layers are the same.

[0226] Figure 8 A schematic diagram of the structure of a query statement generation device based on a large language model provided in an embodiment of the present application, such as Figure 8 As shown, the device of the embodiment of the present application includes: an acquisition module 801, a first processing module 802, a second processing module 803 and a generation module 804. The query statement generation device based on the large language model here can be the above-mentioned processing device itself, or a chip or integrated circuit that realizes the function of the processing device. It should be noted here that the division of the acquisition module 801, the first processing module 802, the second processing module 803 and the generation module 804 is only a division of logical functions, and the two can be physically integrated or independent.

[0227] Wherein, the acquisition module is used to obtain the query text;

[0228] A first processing module is used to perform vectorization processing on the query text to obtain a target vector representation corresponding to the query text;

[0229] A second processing module is used to input the target vector representation to multiple encoders to encode the target vector representation through the weight matrix of each encoder to obtain a context vector corresponding to each encoder, wherein the weight matrix of different encoders is different;

[0230] The generation module is used to generate a target query statement according to the context vectors corresponding to each encoder.

[0231] Optionally, the encoder includes a first self-attention layer and a context vector layer;

[0232] The second processing module is specifically used to: input the target vector representation into the first self-attention layer of multiple encoders, so as to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder, and obtain multiple output vectors of each encoder; input the multiple output vectors of each encoder into the corresponding context vector layer, so as to perform weighted processing on the multiple output vectors through each context vector layer, and obtain the context vector corresponding to each encoder.

[0233] Optionally, before the second processing module is used to input the target vector representation into the first self-attention layer of multiple encoders to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain multiple output vectors of each encoder, the above-mentioned device also includes a training module, which is used to: for each encoder, obtain a training sample corresponding to the encoder, wherein the training sample includes an input data sample and a label sample; input the training sample into the first self-attention layer for forward propagation to obtain an actual output corresponding to the input data sample; obtain a loss gradient according to a preset loss function, actual output and label sample; and use a back-propagation algorithm to update the weight matrix of the first self-attention layer according to the loss gradient until the weight matrix meets the error requirement.

[0234] Optionally, the weight matrix includes self-attention weights, feedforward network weights, layer normalization weights, and position encoding weights.

[0235] Optionally, the generation module includes:

[0236] A concatenation submodule, used to concatenate the context vectors corresponding to each encoder to obtain a target context vector;

[0237] The generation submodule is used to generate a target query statement based on the target context vector.

[0238] Optionally, the generation submodule also includes:

[0239] A decoding submodule, used for decoding the target context vector to obtain a structured query language template;

[0240] The filling submodule is used to fill the structured query language template to obtain the target query statement.

[0241] Optionally, the large language model also includes a query intent identifier, which is located in the decoder of the large language model; the decoding submodule is specifically used to: input the target context vector into the query intent identifier to perform query intent recognition processing through the query intent identifier to obtain at least one key information; determine the structured query language tag of the structured query language template based on the at least one key information.

[0242] Optionally, the decoding submodule is also specifically used to: input the target context vector into a query intent identifier; in the query intent identifier, calculate the target context vector according to a preset weight value, a preset bias value and a preset activation function in a preset intent recognition function to obtain at least one key information.

[0243] Optionally, the large language model further includes a decoder module and a template-guided attention module, the decoder module and the template-guided attention module are located in the decoder of the large language model; the decoding submodule is specifically used for:

[0244] The target context vector is input into a decoder module for decoding, and a plurality of decoder outputs of the decoder module are obtained according to a time step; the plurality of decoder outputs are input into a template-guided attention module; in the template-guided attention module, a predefined structured query language template is obtained; in the template-guided attention module, a structure of the structured query language template is determined according to the predefined structured query language template and the plurality of decoder outputs.

[0245] Optionally, the decoding submodule is also specifically used to: in the template-guided attention module, perform linear transformation processing on each decoder output to obtain multiple linear transformation outputs; in the template-guided attention module, perform linear transformation processing on a predefined structured query language template to obtain a linear transformation template; in the template-guided attention module, perform calculation processing on each linear transformation output and linear transformation template respectively according to a preset calculation function to obtain multiple attention outputs; in the template-guided attention module, perform splicing processing on multiple attention outputs to obtain the structure of the structured query language template.

[0246] Optionally, the decoder module includes a second self-attention layer, a decoding attention layer, and a forward propagation module.

[0247] Optionally, the filling submodule is specifically used to: identify a placeholder in the structured query language template; obtain a preset candidate set, wherein the preset candidate set includes multiple replacement items;

[0248] According to the preset candidate set, the placeholders in the structured query language template are filled to obtain the target query statement.

[0249] Optionally, the filling submodule is further specifically configured to: adopt a beam search algorithm to screen the replacement items in the preset candidate set multiple times, and fill the screened replacement items into the placeholders in the structured query language template until all the placeholders are replaced.

[0250] Optionally, the acquisition module is specifically used to: perform a query in a preset database to obtain a preset candidate set; and / or, obtain historical query data of the user corresponding to the query text, and generate a preset candidate set based on the historical query data.

[0251] Optionally, the first processing module is specifically used to: input the query text into a pre-trained language model to obtain a target vector representation corresponding to the query text through the output of the pre-trained language model.

[0252] Optionally, the large language model also includes an input embedding layer and a first linear function layer; the first processing module is also specifically used to: input the query text into the input embedding layer to output the embedding vector corresponding to the query text through the input embedding layer; input the embedding vector into the first linear function layer to output the target vector representation corresponding to the query text through the first linear function layer.

[0253] refer to Fig. 9 , which shows a schematic diagram of the structure of a query statement generation device 900 based on a large language model suitable for implementing the embodiment of the present disclosure, and the query statement generation device 900 based on a large language model can be a terminal device or a server. The terminal device can include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The query statement generation device based on the large language model shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0254] like Fig. 9 As shown, the query statement generation device 900 based on the large language model may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (Read Only Memory, referred to as ROM) 902 or the program loaded from the storage device 908 to the random access memory (Random Access Memory, referred to as RAM) 903. In the RAM 903, various programs and data required for the operation of the query statement generation device 900 based on the large language model are also stored. The processing device 901, the ROM 902 and the RAM 903 are connected to each other through a bus 904. The input / output (I / O) interface 905 is also connected to the bus 904.

[0255] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the query statement generation device 900 based on a large language model to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The query statement generating apparatus 900 based on a large language model is shown to have various devices, but it should be understood that it is not required to implement or possess all the devices shown, and more or fewer devices may be implemented or possessed alternatively.

[0256] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0257] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0258] The computer-readable medium may be included in the query statement generation device based on a large language model; or may exist independently without being assembled into the query statement generation device based on a large language model.

[0259] The computer-readable medium carries one or more programs. When the one or more programs are executed by the query statement generation device based on the large language model, the query statement generation device based on the large language model executes the method shown in the above embodiment.

[0260] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0261] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0262] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0263] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0264] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0265] The query statement generation device based on a large language model in the embodiment of the present application can be used to execute the technical solutions in the above-mentioned method embodiments of the present application. Its implementation principles and technical effects are similar and will not be repeated here.

[0266] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement any of the above-mentioned query statement generation methods based on a large language model.

[0267] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement any of the above-mentioned query statement generation methods based on a large language model.

[0268] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0269] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0270] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0271] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims. < / column1> When selecting a table name, all possible table names are considered and the table name with the highest probability is selected as the filler. For example, "users" may be the table name with the highest probability. The probability calculation formula is as shown in formula (5): In formula (5), represents the probability calculation result, Q is the question, D is the database, L(D) is the description of D, P is the prompt word, s is the query question, and s is the i-th token of the query question. S606.3. Fill in the first placeholder: Assuming that "users" is the selected table name, the query template now looks like this: SELECT < / column1>

Claims

1. A query statement generation method based on a large language model, characterized in that: The large language model includes a plurality of encoders, and the method includes: Get the query text; Performing vectorization processing on the query text to obtain a target vector representation corresponding to the query text; Inputting the target vector representation to the plurality of encoders to encode the target vector representation using a weight matrix of each encoder to obtain a context vector corresponding to each encoder, wherein the weight matrices of different encoders are different; Generate a target query statement according to the context vectors corresponding to the encoders.

2. The method according to claim 1, characterized in that The encoder comprises a first self-attention layer and a context vector layer; The step of inputting the target vector representation to the plurality of encoders to encode the target vector representation through a weight matrix of each encoder to obtain a context vector corresponding to each encoder includes: Inputting the target vector representation to the first self-attention layer of the plurality of encoders, so as to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder, and obtain a plurality of output vectors of each encoder; The multiple output vectors of each encoder are input into the corresponding context vector layer, so that the multiple output vectors are weighted by each context vector layer to obtain the context vector corresponding to each encoder.

3. The method according to claim 2, characterized in that Before inputting the target vector representation to the first self-attention layer of the plurality of encoders to calculate the target vector representation through the weight matrix of the first self-attention layer of each encoder to obtain a plurality of output vectors of each encoder, the method further includes: For each encoder, obtain a training sample corresponding to the encoder, wherein the training sample includes an input data sample and a label sample; Inputting the training sample into the first self-attention layer for forward propagation to obtain the actual output corresponding to the input data sample; Obtaining a loss gradient according to a preset loss function, the actual output and the label sample; A back-propagation algorithm is used to update the weight matrix of the first self-attention layer according to the loss gradient until the weight matrix meets the error requirement.

4. The method according to claim 3, characterized in that The weight matrix includes self-attention weights, feedforward network weights, layer normalization weights, and position encoding weights.

5. The method according to claim 1, characterized in that The generating a target query statement according to the context vectors corresponding to the encoders includes: Connecting the context vectors corresponding to the encoders to obtain a target context vector; A target query statement is generated according to the target context vector.

6. The method according to claim 5, characterized in that Generating a target query statement according to the target context vector includes: Decoding the target context vector to obtain a structured query language template; The structured query language template is filled to obtain a target query statement.

7. The method according to claim 6, characterized in that The large language model further includes a query intent identifier, and the query intent identifier is located in a decoder of the large language model; The decoding process of the target context vector to obtain a structured query language template includes: Inputting the target context vector into the query intent identifier to perform query intent identification processing through the query intent identifier to obtain at least one key information; A structured query language tag of the structured query language template is determined according to the at least one key information.

8. The method according to claim 7, characterized in that The target context vector is input into the query intent identifier to perform query intent identification processing by the query intent identifier to obtain at least one key information, including: Inputting the target context vector into the query intent identifier; In the query intent identifier, the target context vector is calculated according to a preset weight value, a preset deviation value and a preset activation function in a preset intent identification function to obtain at least one key information.

9. The method according to claim 6, characterized in that The large language model further includes a decoder module and a template-guided attention module, wherein the decoder module and the template-guided attention module are located in the decoder of the large language model; The decoding process of the target context vector to obtain a structured query language template includes: Inputting the target context vector into the decoder module for decoding processing, and obtaining a plurality of decoder outputs of the decoder module according to a time step; inputting the plurality of decoder outputs into the template guided attention module; In the template-guided attention module, a predefined structured query language template is obtained; In the template-guided attention module, a structure of a structured query language template is determined based on the predefined structured query language template and the plurality of decoder outputs.

10. The method according to claim 9, characterized in that The step of determining the structure of the structured query language template in the template-guided attention module according to the predefined structured query language template and the plurality of decoder outputs includes: In the template guided attention module, each of the decoder outputs is subjected to a linear transformation process to obtain a plurality of linear transformation outputs; In the template guided attention module, the predefined structured query language template is subjected to linear transformation processing to obtain a linear transformation template; In the template-guided attention module, each of the linear transformation outputs and the linear transformation template is respectively calculated and processed according to a preset calculation function to obtain a plurality of attention outputs; In the template-guided attention module, the multiple attention outputs are concatenated to obtain the structure of the structured query language template.

11. The method according to claim 9, characterized in that The decoder module includes a second self-attention layer, a decoding attention layer and a forward propagation module.

12. The method according to any one of claims 6 to 11, characterized in that The filling process of the structured query language template to obtain a target query statement includes: Identifying a placeholder in the structured query language template; Obtaining a preset candidate set, wherein the preset candidate set includes a plurality of replacement items; According to the preset candidate set, the placeholders in the structured query language template are filled to obtain a target query statement.

13. The method according to claim 12, characterized in that The filling process of the placeholders in the structured query language template according to the preset candidate set to obtain the target query statement includes: A beam search algorithm is used to screen the replacement items in the preset candidate set for multiple times, and the screened replacement items are filled into the placeholders in the structured query language template until all the placeholders are replaced.

14. The method according to claim 12, characterized in that The obtaining of the preset candidate set includes: Query the preset database to obtain a preset candidate set; And\or, obtaining historical query data of the user corresponding to the query text, and generating a preset candidate set based on the historical query data.

15. The method according to any one of claims 1 to 11, characterized in that The vectorizing the query text to obtain a target vector representation corresponding to the query text includes: The query text is input into a pre-trained language model to obtain a target vector representation corresponding to the query text through an output of the pre-trained language model.

16. The method according to any one of claims 1 to 11, characterized in that The large language model also includes an input embedding layer and a first linear function layer; The vectorizing the query text to obtain a target vector representation corresponding to the query text includes: Inputting the query text into the input embedding layer to output an embedding vector corresponding to the query text through the input embedding layer; The embedding vector is input into the first linear function layer to output a target vector representation corresponding to the query text through the first linear function layer.

17. A query statement generation device based on a large language model, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 16 when executed by a processor.

Citation Information

Patent Citations

  • Deep dynamic context word representation model and method and computer

    CN110222349A

  • Conversation generation method and system, readable storage medium and computer equipment

    CN110321417A

  • Machine translation model training method and device, equipment and storage medium

    CN116681092A

  • Multi-feature fusion problem generation method and system based on two-stage attention

    CN117112758A

Cited By

  • Semantic search method for identifying user intention based on AI and storage medium

    CN119621954A

  • A semantic search method and storage medium based on AI recognition of user intention

    CN119621954B

  • Method, device and equipment for converting natural language into SQL (Structured Query Language) query statement and storage medium

    CN120448406A

  • A method, device and equipment for converting natural language into SQL query statement and storage medium

    CN120448406B

  • Spontaneous transformation method and system between large language model position coding and semantic mechanism

    CN120892947A