Method for finely adjusting TinyLma LLM

Abstract

The invention discloses a method for finely adjusting a TinyLma LLM, and relates to the technical field of data management. Comprising the following steps of: 1, developing a tuning data set required by a TinyLma LLM model; 2, performing fine tuning on the TinyLma LLM model by using the tuning data set; 21, performing fine tuning on the TinyLma LLM model by using a low-rank adaptation LoRA method; and step 22, adjusting the TinyLma LLM model by using a retrieval enhancement generation (RAG) method.

Description

Technical Field

[0001] The invention discloses a method for fine-tuning TinyLlama LLM, and relates to the technical field of data management. Background Art

[0002] SQL (Structured Query Language) is a standardized programming language designed specifically for managing and operating relational databases. It allows users to perform various operations in a declarative way, including data query (DQL-Data Query Language), data manipulation (DML-Data Manipulation Language), data definition (DDL-Data Definition Language), etc.

[0003] Llama LLM: refers to a series of large language models developed by Meta Platform. Llama models are deep learning models tuned on large amounts of text data. Their purpose is to understand and generate human language, and they can perform a variety of natural language processing tasks such as dialogue generation, text completion, translation, question and answer, etc. TinyLlama: is a variant of Llama provided by Hugging Face, which is specially tailored for tasks such as text generation and question and answer. By fine-tuning it on a specific dataset, it can be applied to professional tasks such as natural language generation SQL queries.

[0004] Data Element: refers to data resources that can participate in production activities and bring economic benefits to owners or users in the process of informatization and digitalization. There is currently no method to use data elements to fine-tune TinyLlama so that TinyLlama can more accurately generate professional tasks such as SQL queries from natural language. Summary of the invention

[0005] In view of the problems of the prior art, the present invention provides a method for fine-tuning TinyLlama LLM and a method for querying data using natural language, thereby improving the flexibility and ease of use of data element application.

[0006] The specific scheme proposed by the present invention is:

[0007] The present invention provides a method for fine-tuning TinyLlama LLM, comprising:

[0008] Step 1: Tuning dataset required to develop TinyLlama LLM model:

[0009] Step 11: When using data elements, log each data call information, which includes the called model, method, and executed SQL. Use log data as the tuning data set.

[0010] Step 12: Introduce third-party open source datasets to supplement and optimize the dataset.

[0011] Step 2: Fine-tune the TinyLlama LLM model using the tuning dataset:

[0012] Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method:

[0013] Freeze the basic model weights to reduce computing resource requirements,

[0014] Set fine-tuning parameters: determine learning rate, batch size, training rounds,

[0015] Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results;

[0016] Step 22: Tune the TinyLlama LLM model using the retrieval-augmented generation RAG method:

[0017] Perform preprocessing and text vectorization of the tuning dataset, and map the text of the tuning dataset to the numerical vector space to facilitate subsequent similarity calculations.

[0018] Create an index and perform query conversion: Convert the questions or instructions entered by the user into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector that is most similar to the query vector to complete the adjustment of the TinyLlamaLLM model retrieval and generated sentences.

[0019] Furthermore, when using data elements in step 11 of the method for fine-tuning TinyLlama LLM, data conversion is performed, including: in the data elements, the model represents a table or view in the database, the attribute represents a field in the database, and the attribute type also has a corresponding mapping relationship. The mapping relationship is recorded so that data conversion can be directly completed when data is extracted.

[0020] Furthermore, in step 11 of the method for fine-tuning TinyLlama LLM, ElasticsearchLogstash is used to extract log data, the extracted data is temporarily stored in a front-end library, and data cleaning is performed, including: data cleaning is performed on the data placed in the front-end library, and data cleaning includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

[0021] Furthermore, in step 12 of the method for fine-tuning TinyLlama LLM, the dataset b-mc2 / sql-create-context of the third-party data source HuggingFace is introduced to supplement the tuning dataset, including:

[0022] Translate the question column of the b-mc2 / sql-create-context dataset into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand using the chat template.

[0023] The present invention also provides a device for fine-tuning TinyLlama LLM, including a data set management module and a tuning module.

[0024] The dataset management module develops the tuning datasets required for the TinyLlama LLM model:

[0025] Step 11: When using data elements, log each data call information, which includes the called model, method, and executed SQL. Use log data as the tuning data set.

[0026] Step 12: Introduce third-party open source datasets to supplement and optimize the dataset.

[0027] The tuning module fine-tunes the TinyLlama LLM model using the tuning dataset:

[0028] Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method:

[0029] Freeze the basic model weights to reduce computing resource requirements,

[0030] Set fine-tuning parameters: determine learning rate, batch size, training rounds,

[0031] Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results;

[0032] Step 22: Tune the TinyLlama LLM model using the retrieval-augmented generation RAG method:

[0033] Perform preprocessing and text vectorization of the tuning dataset, and map the text of the tuning dataset to the numerical vector space to facilitate subsequent similarity calculations.

[0034] Create an index and perform query conversion: Convert the questions or instructions entered by the user into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector that is most similar to the query vector to complete the adjustment of the TinyLlamaLLM model retrieval and generated sentences.

[0035] Furthermore, when the data set management module of the device for fine-tuning TinyLlama LLM executes step 11 and utilizes data elements, data conversion is performed, including: in the data elements, the model represents a table or view in the database, the attribute represents a field in the database, and the attribute type also has a corresponding mapping relationship, and the mapping relationship is recorded so that data conversion can be directly completed when data is extracted.

[0036] Furthermore, in the data set management module of the device for fine-tuning TinyLlama LLM, in step 11, Elasticsearch Logstash is used to extract log data, the extracted data is temporarily stored in a front-end library, and data cleaning is performed, including: data cleaning is performed on the data placed in the front-end library, and data cleaning includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

[0037] Furthermore, the data set management module of the device for fine-tuning TinyLlama LLM executes step 12 to introduce the data set b-mc2 / sql-create-context of the third-party data source HuggingFace to supplement the tuning data set, including:

[0038] Translate the question column of the b-mc2 / sql-create-context dataset into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand using the chat template.

[0039] The benefits of the present invention are:

[0040] In the field of data element circulation, the model tuning achieved through two methods, low-rank adaptation (LoRA) and retrieval-augmented generation (RAG), has greatly improved the accuracy of the TinyLlama model in the field of data elements from natural language to SQL. It can further allow users to quickly and accurately query the data they want through natural language, improve the flexibility and ease of use of the system, and reduce the difficulty of using the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the process flow of the present invention.

[0042] Figure 2It is a schematic diagram of an example interface of the chat template code of the present invention. DETAILED DESCRIPTION

[0043] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0044] Example 1

[0045] The present invention provides a method for fine-tuning TinyLlama LLM, comprising:

[0046] Step 1: Tuning dataset required to develop TinyLlama LLM model:

[0047] Step 11: When using data elements, record each data call information through logs. The data call information includes the called model, method, and executed SQL. Use log data as the tuning data set. When using data elements, data conversion can include: in the data elements, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship. Record the mapping relationship so that data conversion can be completed directly during data extraction.

[0048] Elasticsearch Logstash is used to extract log data, temporarily store the extracted data in the front-end library, and perform data cleaning, including: cleaning the data in the front-end library. Data cleaning includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

[0049] Step 12: Introduce third-party open source datasets to supplement and optimize the dataset.

[0050] The b-mc2 / sql-create-context dataset of the third-party data source HuggingFace can be introduced to supplement the tuning dataset, including:

[0051] Translate the question column of the dataset b-mc2 / sql-create-context into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand using the chat template. The code for defining the chat template can be found in Figure 2 .

[0052] Step 2: Fine-tune the TinyLlama LLM model using the tuning dataset:

[0053] Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method:

[0054] Freeze the base model weights to reduce computing resource requirements without making major changes to the pre-model. Even if the fine-tuning effect is not satisfactory, it can be quickly restored.

[0055] Set fine-tuning parameters: determine the learning rate, batch size, and training rounds. Since LoRA has a good amount of parameters, a larger learning rate and more training iterations can be used.

[0056] Verify and test the results: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators such as accuracy, F1 score, etc., and adjust the training strategy or stop training based on the verification results.

[0057] Step 22: Use the retrieval enhancement to generate RAG method to adjust the TinyLlama LLM model. You can use RAG to enhance its retrieval and generation capabilities, thereby improving the quality of the model output. We can use the high-level data processing capabilities of the vector database and similarity search to enhance the retrieval and storage functions of RAG.

[0058] Specifically:

[0059] Perform preprocessing and text vectorization of the tuning dataset, and map the text of the tuning dataset to the numerical vector space to facilitate subsequent similarity calculations.

[0060] Create an index and perform query conversion: Convert the questions or instructions entered by the user into vector form, perform an approximate nearest neighbor ANN (ANN) search in the vector database, and find the document vector that is most similar to the query vector to complete the adjustment of TinyLlama LLM model retrieval and generated sentences.

[0061] Example 2

[0062] The present invention also provides a device for fine-tuning TinyLlama LLM, including a data set management module and a tuning module.

[0063] The dataset management module develops the tuning datasets required for the TinyLlama LLM model:

[0064] Step 11: When using data elements, log each data call information, which includes the called model, method, and executed SQL. Use log data as the tuning data set.

[0065] Step 12: Introduce third-party open source datasets to supplement and optimize the dataset.

[0066] The tuning module fine-tunes the TinyLlama LLM model using the tuning dataset:

[0067] Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method:

[0068] Freeze the basic model weights to reduce computing resource requirements,

[0069] Set fine-tuning parameters: determine learning rate, batch size, training rounds,

[0070] Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results;

[0071] Step 22: Tune the TinyLlama LLM model using the retrieval-augmented generation RAG method:

[0072] Perform preprocessing and text vectorization of the tuning dataset, and map the text of the tuning dataset to the numerical vector space to facilitate subsequent similarity calculations.

[0073] Create an index and perform query conversion: Convert the questions or instructions entered by the user into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector that is most similar to the query vector to complete the adjustment of the TinyLlamaLLM model retrieval and generated sentences.

[0074] As the information interaction and execution process between the modules of the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0075] Similarly, in the field of data element circulation, the device of the present invention realizes model tuning through two methods, low-rank adaptation (LoRA) and retrieval enhanced generation (RAG), which greatly improves the accuracy of the TinyLlama model in the field of data elements from natural language to SQL. It can further allow users to quickly and accurately query the data they want through natural language, improve the flexibility and ease of use of the system, and reduce the difficulty of using the system.

[0076] It should be noted that not all steps and modules in the above-mentioned processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0077] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for fine-tuning TinyLlama LLM, characterized by include: Step 1: Tuning dataset required to develop TinyLlama LLM model: Step 11: When using data elements, log each data call information, which includes the called model, method, and executed SQL. Use log data as the tuning data set. Step 12: Introduce third-party open source datasets to supplement and optimize the dataset. Step 2: Fine-tune the TinyLlama LLM model using the tuning dataset: Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method: Freeze the basic model weights to reduce computing resource requirements, Set fine-tuning parameters: determine learning rate, batch size, training rounds, Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results; Step 22: Tune the TinyLlama LLM model using the retrieval-augmented generation RAG method: Perform preprocessing of the tuning dataset and text vectorization, map the tuning dataset text to the numerical vector space, and facilitate subsequent similarity calculations. Create an index and perform query conversion: Convert the questions or instructions entered by the user into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector that is most similar to the query vector to complete the adjustment of TinyLlama LLM model retrieval and generated sentences.

2. A method for fine-tuning TinyLlama LLM according to claim 1, characterized in that When using data elements in step 11, data conversion is performed, including: in the data elements, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship. The mapping relationship is recorded so that data conversion can be completed directly during data extraction.

3. A method for fine-tuning TinyLlama LLM according to claim 1, characterized in that In step 11, Elasticsearch Logstash is used to extract log data, temporarily store the extracted data in the front-end library, and perform data cleaning, including: cleaning the data in the front-end library. Data cleaning includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

4. A method for fine-tuning TinyLlama LLM according to claim 1, characterized in that In step 12, the b-mc2 / sql-create-context dataset of the third-party data source HuggingFace is introduced to supplement the tuning dataset, including: Translate the question column of the b-mc2 / sql-create-context dataset into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand using the chat template.

5. A device for fine-tuning TinyLlama LLM, characterized in that Including data set management module and tuning module, The dataset management module develops the tuning datasets required for the TinyLlama LLM model: Step 11: When using data elements, log each data call information, which includes the called model, method, and executed SQL. Use log data as the tuning data set. Step 12: Introduce third-party open source datasets to supplement and optimize the dataset. The tuning module fine-tunes the TinyLlama LLM model using the tuning dataset: Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method: Freeze the base model weights to reduce computing resource requirements, Set fine-tuning parameters: determine learning rate, batch size, training rounds, Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results; Step 22: Tune the TinyLlama LLM model using the retrieval-augmented generation RAG method: Perform preprocessing and text vectorization of the tuning dataset, and map the text of the tuning dataset to the numerical vector space to facilitate subsequent similarity calculations. Create an index and perform query conversion: Convert the questions or instructions entered by the user into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector that is most similar to the query vector to complete the adjustment of TinyLlama LLM model retrieval and generated sentences.

6. A device for fine-tuning TinyLlama LLM according to claim 5, characterized in that the data When the set management module executes step 11 and utilizes data elements, data conversion is performed, including: in the data elements, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship. The mapping relationship is recorded so that data conversion can be completed directly during data extraction.

7. A device for fine-tuning TinyLlama LLM according to claim 5, characterized in that the data In step 11, the set management module uses Elasticsearch Logstash to extract log data, temporarily stores the extracted data in the front-end library, and performs data cleaning, including: cleaning the data in the front-end library, which includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

8. The device for fine-tuning TinyLlama LLM according to claim 1, characterized in that the data The data set management module executes step 12 to introduce the b-mc2 / sql-create-context data set of the third-party data source HuggingFace to supplement the tuning data set, including: Translate the question column of the b-mc2 / sql-create-context dataset into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand using the chat template.

Images