Language large model-based statement generation method, apparatus and device, and medium
By storing the correspondence between intents and data tables in the large language model, and combining prompt words for intent recognition and SQL statement generation, the problem of not being able to generate multi-table SQL statements in existing technologies is solved, the generation accuracy is improved, and it is suitable for complex large language model applications.
Patent Information
- Application Number
- CN202411173127.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2026-03-03
AI Technical Summary
Existing technologies cannot effectively generate SQL statements based on multiple data tables, resulting in low generation accuracy.
By pre-storing the correspondence between intents and data tables, and combining it with a large language model for intent recognition and SQL statement generation, the first and second prompt words are used to control the large language model to process the target text and data tables, generating accurate SQL statements.
It enables the generation of SQL statements based on multiple data tables, improving the accuracy of the generation and making it suitable for complex language model application scenarios.
Smart Images
Figure CN121597713A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and medium for sentence generation based on a large language model. Background Technology
[0002] With the development of language big model technology, language big models have been applied to various industries and fields, and converting text into Structured Query Language (SQL) statements is one of them.
[0003] Existing methods for converting text into SQL statements based on large language models are designed for single-table queries, meaning they use only one data table to generate SQL statements. However, as the application of large language models becomes more widespread and the problems they need to handle become more complex, a single data table can no longer meet the current needs for text-to-SQL conversion. Therefore, how to generate SQL statements based on multiple data tables has become a problem that needs to be solved. Summary of the Invention
[0004] This application provides a method, apparatus, device, and medium for generating statements based on a large language model, which solves the problem that existing technologies cannot generate SQL statements based on multiple data tables.
[0005] In a first aspect, embodiments of this application provide a sentence generation method based on a large language model, the method comprising:
[0006] The target text to be processed and the first prompt word pre-saved for the prompt language model to perform intent recognition are input into the language model to obtain the target intent of the target text output by the language model.
[0007] Based on the pre-saved correspondence between intents and data tables, determine at least one target data table corresponding to the target intent;
[0008] The target text, the at least one target data table, and the pre-saved second prompt words used for generating SQL statements in the prompt language model are input into the language model to obtain the target SQL statement corresponding to the target text output by the language model.
[0009] Secondly, embodiments of this application also provide a sentence generation apparatus based on a large language model, the apparatus comprising:
[0010] The intent recognition module is used to input the target text to be processed and the first prompt word pre-saved for intent recognition by the prompt language model into the language model, and obtain the target intent of the target text output by the language model.
[0011] The SQL statement generation module is used to determine at least one target data table corresponding to the target intent based on the pre-saved correspondence between intent and data table; input the target text, the at least one target data table, and the pre-saved second prompt words used for SQL statement generation by the prompt language model into the language model, and obtain the target SQL statement corresponding to the target text output by the language model.
[0012] Thirdly, embodiments of this application also provide an electronic device, which includes at least a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the statement generation method based on the large language model described above.
[0013] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the statement generation method based on the large language model described above.
[0014] In this embodiment, the target text to be processed and a pre-saved first prompt word used for intent recognition by the language model are input into the language model to obtain the target intent of the target text output by the language model. Based on the pre-saved correspondence between intents and data tables, at least one target data table corresponding to the target intent is determined. The target text, at least one target data table, and a pre-saved second prompt word used for SQL statement generation by the language model are input into the language model to obtain the target SQL statement corresponding to the target text output by the language model. In this embodiment, the electronic device pre-saved the correspondence between intents and data tables. The electronic device determines the target intent of the target text to be processed through the language model, thereby determining which target data tables need to be used to generate the SQL statement corresponding to the target text based on the saved correspondence between intents and data tables. Then, the language model is used to generate the SQL statement based on at least one target data table, achieving SQL statement generation based on multiple data tables and ensuring the accuracy of SQL statement generation. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1A schematic diagram illustrating a sentence generation process based on a large language model, provided for an embodiment of this application;
[0017] Figure 2 A schematic diagram illustrating the change in loss value during the fine-tuning of the large language model provided in this application embodiment;
[0018] Figure 3 A flowchart illustrating the fine-tuning of the large language model provided in this application embodiment;
[0019] Figure 4 A schematic diagram illustrating the generation of SQL statements using the large language model provided in this application embodiment;
[0020] Figure 5 A schematic diagram of a sentence generation device based on a large language model provided in this application embodiment;
[0021] Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0023] In order to enable the generation of SQL statements based on multiple data tables using a large control language model and improve the accuracy of SQL statement generation, embodiments of this application provide a statement generation method, apparatus, device, and medium based on a large language model.
[0024] In this embodiment, the target text to be processed and a pre-saved first prompt word for intent recognition in the prompt language model are input into the language model to obtain the target intent of the target text output by the language model; based on the pre-saved correspondence between intent and data table, at least one target data table corresponding to the target intent is determined; the target text, at least one target data table, and a pre-saved second prompt word for SQL statement generation in the prompt language model are input into the language model to obtain the target SQL statement corresponding to the target text output by the language model.
[0025] Figure 1 A schematic diagram of a sentence generation process based on a large language model is provided for embodiments of this application. The process includes:
[0026] S101: Input the target text to be processed and the first prompt word pre-saved for the prompt language model to perform intent recognition into the language model, and obtain the target intent of the target text output by the language model.
[0027] The sentence generation method based on a large language model provided in this application is applied to an electronic device, which may be a PC or a server.
[0028] As the language model is developed, it can query and call dozens or even more data tables. However, in multi-table query scenarios, it is not possible to input all the data tables that the language model can query and call into the language model because the length of the input to the language model is limited. For example, the input limit of the Qwen-14b model is 8,000 tokens. If the input exceeds this length, it will be truncated, which will cause the SQL output by the language model to be unexecutable or unable to retrieve the correct result based on the SQL.
[0029] Based on this, in order to realize the generation of SQL statements based on multiple data tables by the large control language model and improve the accuracy of generating SQL statements, in this embodiment of the application, the electronic device will first perform intent recognition on the target text to be processed to determine the target intent corresponding to the target text.
[0030] Specifically, the electronic device pre-stores first prompt words for intent recognition by the large-scale language model. After receiving a task to generate an SQL statement from the target text, the electronic device retrieves the pre-stored first prompt words for intent recognition and inputs them along with the target text into the large-scale language model. Upon receiving the first prompt words and the target text, the large-scale language model performs intent recognition on the target text based on the first prompt words, determines and outputs the target intent corresponding to the target text. The electronic device then obtains the target intent output by the large-scale language model.
[0031] The first prompt word contains characters related to intent recognition. For example, the first prompt word could be something like, "You are now an intent recognition expert. You need to identify the intent of the user's question." There are no restrictions on this.
[0032] In this embodiment of the application, the target text received by the electronic device can be a question or other text. After receiving the target text, the electronic device can perform standardization processing on the target text, deleting invalid characters from the target text.
[0033] Specifically, the electronic device can pre-store various invalid characters. After receiving the target text, the electronic device identifies and deletes the invalid characters contained in the target text based on the pre-stored invalid characters, and then performs intent recognition and SQL statement generation on the target text.
[0034] S102: Based on the pre-saved correspondence between intents and data tables, determine at least one target data table corresponding to the target intent.
[0035] To accurately determine the data tables required for generating SQL statements for each text and improve the accuracy of SQL statement generation, in this embodiment, the electronic device pre-stores a mapping relationship between intents and data tables. In this mapping relationship, one intent can correspond to one or more data tables, indicating that one or more data tables are required when generating SQL statements for the text containing that intent. Furthermore, this mapping relationship can store the correspondence between intents and data table identifiers. These identifiers can be the name, number, or other characters that allow locating the data table, and each data table identifier is unique.
[0036] Specifically, after determining the target intent of the target text through the large model, the electronic device can determine the identifier of at least one target data table corresponding to the target intent based on the pre-saved correspondence between intent and data table, and determine the data table corresponding to the identifier of the at least one target data table as the target data table required to generate the SQL statement corresponding to the target text.
[0037] Table 1 shows an example of the correspondence between intents and data tables provided in the embodiments of this application:
[0038] intention surface Search for student name and grades Student table, Score table meaning Figure 2 Tables 3 and 4 …… ……
[0039] Table 1
[0040] As shown in Table 1, if the target intent is "to query student names and scores", the electronic device determines at least one target data table corresponding to the target intent as the Student table and the Score table based on the correspondence between intent and data tables; if the target intent is "to query student names and scores", the electronic device determines at least one target data table corresponding to the target intent as the Student table and the Score table. Figure 2 If the electronic device determines, based on the correspondence between the intent and the data table, at least one target data table corresponding to the target intent is Table 3 and Table 4.
[0041] S103: Input the target text, the at least one target data table, and the pre-saved second prompt words used for generating SQL statements in the prompt language model into the language model, and obtain the target SQL statement corresponding to the target text output by the language model.
[0042] In this embodiment of the application, the electronic device also pre-stores a second prompt word for the prompt language big model to generate SQL statements. The electronic device can control the language big model to generate SQL statements for the target text based on the second prompt word and at least one predetermined target data table.
[0043] Specifically, after the electronic device determines at least one target data table corresponding to the intent of the target text, it inputs the target text, the at least one data table corresponding to the target intent of the target text, and a pre-saved second prompt word used for SQL statement generation by the language model into the language model. Upon receiving the input from the electronic device, the language model determines, based on the second prompt word, that SQL statement generation is required. Then, based on the at least one data table, the language model generates and outputs the target SQL statement corresponding to the target text. The electronic device then obtains the target SQL statement output by the language model.
[0044] The second prompt word contains characters related to SQL statement generation. For example, the second prompt word could be something like "You are now a database expert, and you need to generate SQL statements from the text," and there are no restrictions on this.
[0045] In this embodiment, the electronic device pre-stores the correspondence between intents and data tables. The electronic device determines the target intent of the target text to be processed by having the language big model determine the target intent of the target text by having the language big model determine the target intent or data tables to be used to generate the SQL statement based on the stored correspondence between intents and data tables. Then, the language big model generates the SQL statement based on at least one target data table, thereby realizing the generation of SQL statements based on multiple data tables and ensuring the accuracy of SQL statement generation.
[0046] To enable the large-scale control language model to generate SQL statements based on multiple data tables and improve the accuracy of SQL statement generation, based on the above embodiments, in this embodiment, the step of inputting the target text to be processed and the pre-saved first prompt word used for intent recognition by the large-scale control language model into the large-scale control language model includes:
[0047] Obtain a pre-saved intent recognition prompt template, wherein the intent recognition prompt template carries the first prompt word;
[0048] The target text is added to the first preset field of the intent recognition prompt template, and the completed intent recognition prompt template is input into the language big model.
[0049] In order to improve the accuracy of intent recognition by the language big model when the electronic device controls the language big model, in this embodiment of the application, in addition to inputting the first prompt word and target text used to prompt the language big model to perform intent recognition, the electronic device can also input a list of all intents into the language big model, and can also input example text and corresponding example intents.
[0050] The intent list contains all intents that are pre-stored in the electronic device and correspond to a data table.
[0051] In order to manage the input of the language big model in a unified manner, in this embodiment of the application, an intent recognition prompt template is pre-configured in the electronic device. The intent recognition prompt template carries a first prompt word for prompting the language big model to perform intent recognition, an intent list, sample text and corresponding sample intent, and the intent recognition prompt template also contains a first preset field to be filled in, which is used to fill in the target text to be recognized.
[0052] For example, the intent recognition prompt template could be: "You are now an intent recognition expert. You need to identify the user's question based on the user's question. The complete list of intents is as follows: ['Query student name and grades', 'Intent', ... Figure 2 ','meaning Figure 3 '、……、'Intent n']; Example: Input: Query Xiaoming's Chinese score, Output: Query student's name and score; Input: First preset field.
[0053] Based on this, after acquiring the target text to be processed, the electronic device obtains a pre-saved intent recognition prompt template and identifies the first preset field in the intent recognition prompt template used to fill in the text to be recognized. The electronic device adds the target text to the first preset field and inputs the completed intent recognition prompt template into the language big data model for intent recognition.
[0054] For example, if the electronic device receives the target text "Query Lucy's math score", and adds this target text to the first preset field of the intent recognition prompt template, then the completed intent recognition prompt template will be: "You are now an intent recognition expert. You need to identify the user's question based on the user's question. The complete list of intents is as follows: ['Query student name and score', '... ... Figure 2 ','meaning Figure 3 ['、……、'Intention n']; Example: Input: Query Xiaoming's Chinese score, Output: Query student's name and score; Input: Query Lucy's math score.
[0055] To enable the large-scale control language model to generate SQL statements based on multiple data tables and improve the accuracy of SQL statement generation, based on the above embodiments, in this embodiment, the step of inputting the target text, the at least one target data table, and the pre-saved second prompt words used for SQL statement generation by the large-scale control language model into the large-scale control language model includes:
[0056] Obtain a pre-saved SQL statement generation prompt template, wherein the SQL statement generation prompt template carries the second prompt word;
[0057] Add the target text to the second preset field of the SQL statement generation prompt template, and add the table structure corresponding to the at least one target data table to the third preset field of the SQL statement generation prompt template;
[0058] Input the completed SQL statement prompt template into the language model.
[0059] In order to improve the accuracy of SQL statement generation by the language big model when the electronic device controls the language big model, in this embodiment of the application, in addition to inputting the second prompt word, at least one target data table and target text into the language big model, the electronic device can also input sample text and corresponding sample SQL statements into the language big model.
[0060] To manage the input of the large language model uniformly, in this embodiment, an SQL statement generation prompt template is pre-configured in the electronic device. This template includes a second prompt word, sample text, and a corresponding sample SQL statement for prompting the large language model to generate SQL statements. Furthermore, the template contains a second preset field and a third preset field to be filled in. The second preset field is used to fill in the target text for SQL statement generation, and the third preset field is used to fill in information about at least one target data table used in SQL statement generation.
[0061] For example, the SQL statement generation prompt template could be: "You are now a database expert, and you need to generate a query statement based on the question. The tables you need and their structures are as follows: Third preset field; Example: Input: Query Xiaoming's Chinese score, Output: select t1.name,t2.score from student t1left join score t2 ont1.id=t2.id where t1.name='Xiaoming' and t2.subject='Chinese'; Input: Second preset field."
[0062] Based on this, after acquiring the target text to be processed, the electronic device obtains a pre-saved SQL statement generation prompt template and identifies a second preset field in the template for filling in the text to be generated into the SQL statement, as well as a third preset field for filling in information about at least one target data table used in generating the SQL statement. The electronic device adds the target text to the second preset field, adds the table structure of at least one target data table to the third preset field, and inputs the completed SQL statement generation prompt template into the language model to generate the SQL statement.
[0063] For example, if the target text received by the electronic device is "Query Lucy's math score", and the electronic device adds this target text to the second preset field of the SQL statement generation prompt template, then the completed intent recognition prompt template will be: "You are now a database expert. You need to generate a query statement based on the question. The tables you need and their structures are as follows: student table, id student ID, name student name; score table, id student ID, score student score, subject subject; Example: Input: Query Xiaoming's Chinese score, Output: select t1.name,t2.score from student t1left join score t2 on t1.id=t2.id where t1.name='Xiaoming' and t2.subject='Chinese'; Input: Query Lucy's math score."
[0064] To achieve the goal of generating SQL statements based on multiple data tables using a large control language model and improve the accuracy of SQL statement generation, the fine-tuning process of the large language model in this embodiment, based on the above embodiments, includes:
[0065] Obtain the pre-saved first sample text and corresponding sample intent for intent recognition, and use the first sample text, the sample intent and the first prompt word as the first sample for fine-tuning the language model;
[0066] Obtain the pre-saved second sample text for generating SQL statements, along with at least one corresponding sample data table and sample SQL statement, and use the second sample text, the at least one sample data table, the sample SQL statement, and the second prompt word as the second sample for fine-tuning the language model.
[0067] The first sample and the second sample are used as the fine-tuning dataset, and the language model is fine-tuned according to the fine-tuning dataset and the preset LoRa fine-tuning algorithm.
[0068] Current technologies rely on fine-tuning large language models using open-source datasets. This makes it difficult to adapt these models to specific enterprise-specific data, significantly reducing accuracy. Furthermore, the input text in open-source datasets is typically in English, while the table structures and fields in domestic enterprise databases are tailored to their specific business needs and use cases. These use cases often involve Chinese input, and queries may involve cascading queries across multiple tables. In such cases, existing language model fine-tuning methods cannot adequately meet the needs of domestic enterprises.
[0069] To address the need for cascading queries across multiple data tables for specific domains of Chinese data, this application's embodiments fine-tune the language model for the internal data of domestic enterprises, enabling the language model to be well-suited for the internal data of domestic enterprises, thereby improving the accuracy of the language model in querying internal data.
[0070] In this embodiment, text carried in the enterprise's domain data is obtained, and this text is used as sample text. The text is then manually labeled. Specifically, the sample text can be divided into two groups. One group is labeled with intent to determine the corresponding sample intent, and the sample text in this group is used as the first sample text for intent recognition. The other group is labeled with SQL statements to determine at least one corresponding sample data table and sample SQL statement, and the sample text in this group is used as the second sample text for SQL statement generation.
[0071] The electronic device receives input first sample text for intent recognition and the corresponding sample intent, and uses the saved first prompt word for intent recognition in the large-scale language model, the first sample text, and the corresponding sample intent as the first sample for fine-tuning the large-scale language model. Specifically, the electronic device can add the first sample text to an intent recognition prompt template carrying the first prompt word, and use the added intent recognition prompt template and sample intent as the first sample. The process of adding the first sample text to the intent recognition prompt template is consistent with the process of adding the target text to the intent recognition prompt template in the above embodiment, and will not be described again here.
[0072] Furthermore, the electronic device receives the input second sample text for SQL statement generation, along with at least one corresponding sample data table and a sample SQL statement. It then uses the saved second prompt word for SQL statement generation from the large-scale language model, the second sample text, and the corresponding at least one sample data table and sample SQL statement as a second sample for fine-tuning the large-scale language model. Specifically, the electronic device can add the second sample text and the corresponding at least one sample data table to an SQL statement generation prompt template carrying the second prompt word, and use the completed SQL statement generation prompt template and sample SQL statement as the second sample. The process of adding the second sample text and the corresponding at least one sample data table to the SQL statement generation prompt template is consistent with the process of adding the target text and at least one target data table to the SQL statement generation prompt template in the above embodiment, and will not be described again here.
[0073] In this embodiment of the application, after determining a first sample for fine-tuning the intent recognition capability of the language big model and a second sample for fine-tuning the SQL statement generation capability of the language big model, the electronic device uses the first sample and the second sample as a fine-tuning dataset, and fine-tunes the language big model according to the fine-tuning dataset and the preset LoRa fine-tuning algorithm.
[0074] Specifically, by setting different learning rates and other parameters, the algorithm calculates the loss value of the loss function based on the sample intent or sample SQL statement corresponding to the samples in the fine-tuning dataset and the prediction intent or prediction SQL statement determined by the language big model. The algorithm then adjusts the values of the set learning rates and other parameters based on the loss value, iterating repeatedly until the loss value is fine-tuned to a very small value. At this point, the fine-tuning of the language big model is considered complete.
[0075] Figure 2 This is a schematic diagram illustrating the change in loss value during the fine-tuning of the large language model provided in this application embodiment. Figure 2 As shown, the loss value gradually decreases with the increase of the number of iterations.
[0076] Figure 3 The flowchart for fine-tuning the large language model provided in this application embodiment is as follows: Figure 3As shown, the process includes: determining a first sample (intent recognition data) for fine-tuning the intent recognition capability of the large language model, and a second sample (text-to-SQL data) for fine-tuning the SQL statement generation capability of the large language model; performing mixed task fine-tuning of the large language model using the first and second samples; and deploying the fine-tuned large language model. The first sample text in the first sample and the second sample text in the second sample are manually annotated, which requires annotation based on the company's business data and query data scenarios. The intent recognition data is used to fine-tune the intent recognition capability of the large model, allowing the large model to know which tables to query.
[0077] For example, if the first sample text is "Query Xiaoming's Chinese score", and the manually labeled intent corresponding to this first sample text is "Query student name and score", then the electronic device would construct the corresponding first sample as: {"instruction": "You are now an intent recognition expert. You need to identify the user's intent based on the question. All intents are ['Query student name and score', 'intent', 'query student name and score', 'query student name and score', 'intent ...intent Figure 2 ','meaning Figure 3 '……], "input": "Query Xiaoming's Chinese score", "output": "Query student name and score"}, this first sample is the intent recognition data.
[0078] The second sample text is "Query Xiaoming's Chinese score". The manually annotated SQL statement corresponding to this second sample text is "select t1.name,t2.score from student t1 left join score t2 on t1.id=t2.id where t1.name='Xiaoming'and t2.subject='Chinese'". The electronic device then constructs the corresponding second sample as follows: {"instruction": "You are now a database expert. You need to generate a query statement based on the problem. The tables and their structures are as follows: student table, id student id, name student name; score table, id student id, score student score, subject subject", "input": "Query Xiaoming's Chinese score", "output": "select t1.name,t2.score from student t1 left join score t2 on t1.id=t2.id where t1.name='Xiaoming'and t2.subject='Chinese'"}. This second sample is the text-to-SQL data.
[0079] To achieve the generation of SQL statements based on multiple data tables using a large control language model and improve the accuracy of SQL statement generation, in this embodiment of the application, the method further includes, based on the above embodiments:
[0080] Deploy the finely tuned large language model into the llama-factory framework.
[0081] In this embodiment of the application, after the large language model is fine-tuned, the electronic device can deploy the fine-tuned large language model to the llama-factory framework.
[0082] To enable the large-scale language model to generate SQL statements based on multiple data tables and improve the accuracy of SQL statement generation, based on the above embodiments, in this embodiment of the application, before inputting the target text to be processed and the pre-saved intent recognition prompts into the large-scale language model, the method further includes:
[0083] Receive input SQL statement generation instructions carrying the target text;
[0084] Obtain the pre-saved Uniform Resource Locator (URL) of the language big model used for SQL statement generation, and invoke the language big model based on the URL.
[0085] In this embodiment of the application, after the language big model is deployed on the llama-factory framework, the IP and port of the language big model will be exposed to the outside world, that is, the uniform resource locator (URL) of the language big model will be exposed. The electronic device saves the URL of the language big model, so that the electronic device can directly call the language big model to perform the task of converting text to SQL statements according to the URL.
[0086] Based on this, in this embodiment of the application, the electronic device receives an input SQL statement generation instruction carrying the target text; the electronic device obtains the URL of the saved language big model for SQL statement generation, and calls the language big model according to the URL, so that the language big model generates the SQL statement corresponding to the target text.
[0087] Figure 4 This is a schematic diagram illustrating the generation of SQL statements using the large language model provided in this application embodiment. Figure 4As shown, the process includes: receiving the input target text; performing intent recognition on the target text based on the language big model and pre-saved first prompt words used for intent recognition in the language big model; determining the data table corresponding to the intent of the target text according to the correspondence between the saved intent and the data table; and generating an SQL statement for the target text based on the language big model and pre-saved second prompt words used for SQL statement generation in the language big model.
[0088] In order to enable the large control language model to generate SQL statements based on multiple data tables and improve the accuracy of SQL statement generation, based on the above embodiments, in this embodiment of the application, the large language model is the Qwen model.
[0089] In this embodiment of the application, the language model for generating SQL statements can be the Qwen model.
[0090] Based on the above embodiments, the SQL statement generation process of this application will be described in conjunction with a specific embodiment:
[0091] 1. Intent recognition.
[0092] The electronic device accesses the language model service via SQL through the deployed language model, allowing the model to perform an intent recognition task. The question to be processed is "Query Lucy's math score." Therefore, the input to the language model during the intent recognition task is: "You are now an intent recognition expert. You need to identify the user's question based on the question. The complete list of intents is as follows: ['Query student name and score', 'Intent']..." Figure 2 ','meaning Figure 3 '...', Example: Input: Query Xiaoming's Chinese score, Output: Query student name and score, Input: 'Query Lucy's math score', The target intent output by the language model is "Query student name and score". The electronic device determines that at least one target data table corresponding to this target intent is the student table and the score table based on the correspondence between the saved intent and the data table.
[0093] 2. SQL generation
[0094] The electronic device constructs prompts for a language model based on at least one target data table. Specifically, it organizes the table structures of the `student` and `score` tables into prompts. The resulting prompt is: "You are now a database expert. You need to generate a query statement based on the question. The tables you need and their structures are as follows: `student` table: `id` (student ID), `name` (student name); `score` table: `id` (student ID), `score` (student score), `subject` (subject). Example: Input: Query Xiaoming's Chinese score; Output: `select t1.name, t2.score from student t1 left join score t2 on t1.id = t2.id where t1.name = 'Xiaoming' and t2.subject = 'Chinese'; Input: Query Lucy's math score;` The SQL statement output by the language model is: `select t1.name, t2.score from student t1 left join score t2 on t1.id = t2.id where t1.name = 'lucy' and t2.subject = 'math'.`
[0095] Based on the above embodiments, Figure 5 A schematic diagram of a sentence generation device based on a large language model is provided for embodiments of this application. The device includes:
[0096] The intent recognition module 501 is used to input the target text to be processed and the first prompt word pre-saved for intent recognition of the prompt language model into the language model, and obtain the target intent of the target text output by the language model.
[0097] The SQL statement generation module 502 is used to determine at least one target data table corresponding to the target intent based on the pre-saved correspondence between intent and data table; input the target text, the at least one target data table and the pre-saved second prompt words used for SQL statement generation by the prompt language model into the language model, and obtain the target SQL statement corresponding to the target text output by the language model.
[0098] In one possible implementation, the intent processing module 501 is specifically used to obtain a pre-saved intent recognition prompt template, wherein the intent recognition prompt template carries the first prompt word; add the target text to a first preset field of the intent recognition prompt template, and input the added intent recognition prompt template into the language big model.
[0099] In one possible implementation, the SQL statement generation module 502 is specifically used to obtain a pre-saved SQL statement generation prompt template, wherein the SQL statement generation prompt template carries the second prompt word; add the target text to a second preset field of the SQL statement generation prompt template, add the table structure corresponding to the at least one target data table to a third preset field of the SQL statement generation prompt template; and input the completed SQL statement generation prompt template into the language big model.
[0100] In one possible implementation, the device further includes:
[0101] The fine-tuning module 503 is used to acquire a pre-saved first sample text for intent recognition and the corresponding sample intent, and use the first sample text, the sample intent, and the first prompt word as the first sample for fine-tuning the language model; acquire a pre-saved second sample text for SQL statement generation and the corresponding at least one sample data table and sample SQL statement, and use the second sample text, the at least one sample data table, the sample SQL statement, and the second prompt word as the second sample for fine-tuning the language model; use the first sample and the second sample as the fine-tuning dataset, and fine-tune the language model according to the fine-tuning dataset and the preset LoRa fine-tuning algorithm.
[0102] In one possible implementation, the fine-tuning module 503 is also used to deploy the fine-tuned language model to the llama-factory framework.
[0103] In one possible implementation, the intent recognition module 501 is further configured to receive an input SQL statement generation instruction carrying the target text; obtain a pre-saved Uniform Resource Locator (URL) for the language big model used for SQL statement generation; and invoke the language big model according to the URL.
[0104] In one possible implementation, the large language model is the Qwen model.
[0105] Based on the above embodiments, this application also provides an electronic device. Figure 6 This application provides a schematic diagram of an electronic device structure, such as... Figure 6 As shown, it includes: processor 601, communication interface 602, memory 603 and communication bus 604, wherein processor 601, communication interface 602 and memory 603 communicate with each other through communication bus 604.
[0106] The memory 603 stores a computer program, which, when executed by the processor 601, causes the processor 601 to perform the steps of the statement generation method based on the large language model provided in the above embodiments.
[0107] Since the principle behind the problem-solving by the aforementioned electronic device is similar to that of the sentence generation method based on a large language model, the implementation of the aforementioned electronic device can be found in the embodiments of the method, and repeated details will not be elaborated here.
[0108] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. Communication interface 602 is used for communication between the aforementioned electronic device and other devices. The memory can include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.
[0109] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0110] Based on the above embodiments, this invention also provides a computer-readable storage medium storing a computer program executable by a processor. When the program runs on the processor, it causes the processor to implement the statement generation method steps based on the large language model provided in the above embodiments.
[0111] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0112] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0114] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0115] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A sentence generation method based on a large language model, characterized in that, The method includes: The target text to be processed and the first prompt word pre-saved for the prompt language model to perform intent recognition are input into the language model to obtain the target intent of the target text output by the language model. Based on the pre-saved correspondence between intents and data tables, determine at least one target data table corresponding to the target intent; The target text, the at least one target data table, and the pre-saved second prompt words used for generating SQL statements in the prompt language model are input into the language model to obtain the target SQL statement corresponding to the target text output by the language model.
2. The method according to claim 1, characterized in that, The step of inputting the target text to be processed and the pre-saved first prompt word used for intent recognition in the large language model includes: Obtain a pre-saved intent recognition prompt template, wherein the intent recognition prompt template carries the first prompt word; The target text is added to the first preset field of the intent recognition prompt template, and the completed intent recognition prompt template is input into the language big model.
3. The method according to claim 1, characterized in that, The step of inputting the target text, the at least one target data table, and the pre-saved second prompt word used for generating SQL statements for the prompt language model into the language model includes: Obtain a pre-saved SQL statement generation prompt template, wherein the SQL statement generation prompt template carries the second prompt word; Add the target text to the second preset field of the SQL statement generation prompt template, and add the table structure corresponding to the at least one target data table to the third preset field of the SQL statement generation prompt template; Input the completed SQL statement prompt template into the language model.
4. The method according to claim 1, characterized in that, The fine-tuning process of the large language model includes: Obtain the pre-saved first sample text and corresponding sample intent for intent recognition, and use the first sample text, the sample intent and the first prompt word as the first sample for fine-tuning the language model; Obtain the pre-saved second sample text for generating SQL statements, along with at least one corresponding sample data table and sample SQL statement, and use the second sample text, the at least one sample data table, the sample SQL statement, and the second prompt word as the second sample for fine-tuning the language model. The first sample and the second sample are used as the fine-tuning dataset, and the language model is fine-tuned according to the fine-tuning dataset and the preset LoRa fine-tuning algorithm.
5. The method according to claim 4, characterized in that, The method further includes: Deploy the finely tuned large language model into the llama-factory framework.
6. The method according to claim 1, characterized in that, Before inputting the target text to be processed and the pre-saved intent recognition prompts into the language large model, the method further includes: Receive input SQL statement generation instructions carrying the target text; Obtain the pre-saved Uniform Resource Locator (URL) of the language big model used for SQL statement generation, and invoke the language big model based on the URL.
7. The method according to any one of claims 1-6, characterized in that, The large-scale language model is the Qwen model.
8. A sentence generation device based on a large language model, characterized in that, The device includes: The intent recognition module is used to input the target text to be processed and the first prompt word pre-saved for intent recognition by the prompt language model into the language model, and obtain the target intent of the target text output by the language model. The SQL statement generation module is used to determine at least one target data table corresponding to the target intent based on the pre-saved correspondence between intent and data table; input the target text, the at least one target data table, and the pre-saved second prompt words used for SQL statement generation by the prompt language model into the language model, and obtain the target SQL statement corresponding to the target text output by the language model.
9. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the statement generation method based on a large language model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the statement generation method based on a large language model as described in any one of claims 1-7.