A method of fine-tuning a TinyLlama LLM

Abstract

The present invention discloses a method for fine-tuning TinyLlama LLM, which relates to the field of data management technology; the method comprises: step 1: developing a tuning data set required for the TinyLlama LLM model; step 2: fine-tuning the TinyLlama LLM model using the tuning data set; step 21: fine-tuning the TinyLlama LLM model using a low-rank adaptation LoRA method; and step 22: adjusting the TinyLlama LLM model using a retrieval enhancement generation RAG method.

Description

TECHNICAL FIELD

[0001] The application discloses a method for fine-tuning TinyLlama LLM, and relates to the technical field of data management. BACKGROUND

[0002] SQL (Structured Query Language) is a standardized programming language designed specifically for managing and manipulating relational databases. It allows users to perform various operations in a declarative manner, including data querying (DQL-Data Query Language), data manipulation (DML-Data Manipulation Language), and data definition (DDL-Data Definition Language).

[0003] Llama LLM refers to a series of large language models developed by Meta Platform. Llama models are deep learning models that are fine-tuned on large amounts of text data, with the goal of understanding and generating human language. They can perform various natural language processing tasks such as dialogue generation, text completion, translation, and question answering.

[0004] Data elements refer to data resources that can participate in production activities and bring economic benefits to owners or users during the informationization and digitization process. Currently, there is no method for fine-tuning TinyLlama using data elements to enable TinyLlama to more accurately generate SQL queries and other professional tasks in natural language. SUMMARY

[0005] The application provides a method for fine-tuning TinyLlama LLM and a method for querying data using natural language, thereby improving the flexibility and ease of use of data element applications.

[0006] The specific scheme provided by the application is as follows:

[0007] The application provides a method for fine-tuning TinyLlama LLM, comprising:

[0008] Step 1: Develop the fine-tuning data set required for the TinyLlama LLM model:

[0009] Step 11: When utilizing data elements, log each data call information, including the called model, method, and executed SQL, using log data as the tuning dataset,

[0010] Step 12: Introduce third-party open-source datasets to supplement the tuning dataset,

[0011] Step 2: Fine-tune the TinyLlama LLM model using the tuning dataset:

[0012] Step 21: Fine-tune the TinyLlama LLM model using the low-rank adaptation LoRA method:

[0013] Freeze the base model weights to reduce computational resource requirements,

[0014] Set fine-tuning parameters: determine learning rate, batch size, and training rounds,

[0015] Perform effectiveness verification and testing: regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust training strategies or stop training based on validation results;

[0016] Step 22: Adjust the TinyLlama LLM model using the retrieval-augmented generation RAG method:

[0017] Perform tuning dataset preprocessing and text vectorization to map tuning dataset text to numerical vector space for subsequent similarity calculations,

[0018] Create indexes and perform query conversion: convert user input questions or instructions into vector form, perform approximate nearest neighbor ANN search in the vector database, find the most similar document vectors to the query vector, and use them to complete TinyLlama LLM model retrieval and generated statement adjustment.

[0019] Further, in step 11 of the method for fine-tuning TinyLlama LLM, when utilizing data elements, data conversion is performed, including: in the data elements, the model represents tables or views in the database, the attributes represent fields in the database, and the attribute types also have corresponding mapping relationships. Record the mapping relationships to facilitate data extraction and complete data conversion directly.

[0020] Further, in step 11 of the method for fine-tuning TinyLlama LLM, Elasticsearch Logstash is used to extract log data, and the extracted data is temporarily stored in the front-end library and subjected to data cleaning, including: data cleaning of data stored in the front-end library, including removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

[0021] Further, the step 12 of the method for fine-tuning the TinyLlama LLM introduces a third-party data source HuggingFace's dataset b-mc2 / sql-create-context to supplement the tuning dataset, including:

[0022] The question column of the dataset b-mc2 / sql-create-context is translated into Chinese data, the Chinese data is used to supplement the tuning dataset, and a chat template is defined to convert the Chinese data into a format that TinyLlama can understand.

[0023] The application also provides a device for fine-tuning the TinyLlama LLM, including a dataset management module and a tuning module,

[0024] The dataset management module develops the tuning dataset required by the TinyLlama LLM model:

[0025] Step 11: When using data elements, record each data call information through logs, including the called model, method and executed SQL, and use log data as a tuning dataset,

[0026] Step 12: Introduce a third-party open source dataset to supplement the tuning dataset,

[0027] The tuning module uses the tuning dataset to fine-tune the TinyLlama LLM model:

[0028] Step 21: Fine-tune the TinyLlama LLM model using the low-rank adaptive LoRA method:

[0029] Freeze the base model weights to reduce the demand for computing resources,

[0030] Set the fine-tuning parameters: determine the learning rate, batch size, and training rounds,

[0031] Effect verification and testing: regularly evaluate the model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training according to the validation results;

[0032] Step 22: Adjust the TinyLlama LLM model using the retrieval-enhanced generation RAG method:

[0033] Preprocess the tuning dataset and text vectorization, map the tuning dataset text to a numerical vector space for subsequent similarity calculation,

[0034] Index is created, and query conversion is carried out: the question or instruction input by the user is converted into a vector form, an approximate nearest neighbor (ANN) search is carried out in the vector database, the document vector most similar to the query vector is found, and is used to complete the TinyLlama LLM model retrieval and the adjustment of the generated sentence.

[0035] Further, the data set management module of the device for fine-tuning TinyLlama LLM performs step 11 when using data elements, and carries out data conversion, including: in the data elements, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship; the mapping relationship is recorded, so that data conversion is directly completed when data extraction is carried out.

[0036] Further, the data set management module of the device for fine-tuning TinyLlama LLM performs step 11, uses Elasticsearch Logstash to extract log data, temporarily stores the extracted data in a front-end library, and carries out data cleaning, including: the data stored in the front-end library is subjected to data cleaning, and the data cleaning includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

[0037] Further, the data set management module of the device for fine-tuning TinyLlama LLM performs step 12, introduces the third-party data source HuggingFace's data set b-mc2 / sql-create-context to supplement the tuning data set, including:

[0038] The question column of the data set b-mc2 / sql-create-context is translated into Chinese data, the Chinese data is used to supplement the tuning data set, and a chat template is defined, the Chinese data is converted into a format that can be understood by TinyLlama by using the chat template.

[0039] The present application has the advantages that:

[0040] In the field of data element circulation, the model tuning realized by the two methods of low rank adaptation (LoRA) and retrieval augmented generation (RAG) greatly improves the accuracy of the TinyLlama model in converting natural language to SQL in the field of data elements. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 It is a method flow schematic diagram of the present application.

[0042] Figure 2Is the chat template code example interface schematic diagram of the present application. DETAILED DESCRIPTION

[0043] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not as a limitation on the present application.

[0044] Embodiment 1

[0045] The present application provides a method for fine-tuning TinyLlama LLM, comprising:

[0046] Step 1: Develop the tuning dataset required for the TinyLlama LLM model:

[0047] Step 11: When using data elements, log each data call information, including the called model, method, and executed SQL, using log data as the tuning dataset. When using data elements, data conversion can be performed, which can include: in the data element, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship. Record the mapping relationship so that data conversion can be completed directly during data extraction.

[0048] And use Elasticsearch Logstash to extract log data, temporarily store the extracted data in the front-end library, and perform data cleaning, including: data cleaning of data placed in the front-end library, data cleaning includes removing unnecessary information, correcting spelling errors, and filtering out low-quality data.

[0049] Step 12: Introduce third-party open-source datasets to supplement the tuning dataset.

[0050] The third-party data source HuggingFace dataset b-mc2 / sql-create-context can be introduced to supplement the tuning dataset, including:

[0051] Translate the question column of the dataset b-mc2 / sql-create-context into Chinese data, supplement the tuning dataset with Chinese data, and define a chat template to convert the Chinese data into a format that TinyLlama can understand. The code for defining the chat template can be referred to Figure 2 .

[0052] Step 2: Fine-tune the TinyLlama LLM model using the tuning dataset:

[0053] Step 21: Fine-tune the TinyLlama LLM model using the low-rank adaptation LoRA method:

[0054] Freeze the base model weights, reduce the computational resource requirements, and do not make large modifications to the pre-model. Even if the fine-tuning effect is not satisfactory, it can be quickly recovered.

[0055] Set fine-tuning parameters: determine learning rate, batch size, training rounds, and since the parameter quantity of LoRA is good, a larger learning rate and more training iterations can be used.

[0056] Perform effect verification and testing: regularly evaluate model performance on the validation set, monitor overfitting and performance indicators such as accuracy and F1 score, and adjust the training strategy or stop training according to the validation results.

[0057] Step 22: Adjust the TinyLlama LLM model using the retrieval enhancement generation RAG method, which can use the RAG method to improve its retrieval and generation capabilities, thereby improving the quality of model output. We can use the high-dimensional data processing capabilities of the vector database and similarity search to perform RAG retrieval and storage functions.

[0058] Specifically:

[0059] Perform tuning dataset preprocessing and text vectorization to map tuning dataset text to numerical vector space for subsequent similarity calculation,

[0060] Create an index and perform query conversion: convert user input questions or instructions into vector form, perform Approximate Nearest Neighbor (ANN) search in the vector database, find the most similar document vector to the query vector, and use it to complete the TinyLlama LLM model retrieval and generation statement adjustment.

[0061] Embodiment 2

[0062] The application also provides a device for fine-tuning a TinyLlama LLM, comprising a dataset management module and a tuning module,

[0063] The dataset management module develops the tuning dataset required for the TinyLlama LLM model:

[0064] Step 11: When using data elements, log each data call information, including the called model, method, and executed SQL, and use log data as the tuning dataset,

[0065] Step 12: Introduce third-party open-source datasets to supplement the tuning dataset,

[0066] The tuning module fine-tunes the TinyLlama LLM model using the tuning dataset:

[0067] Step 21: Fine-tune the TinyLlama LLM model using the low-rank adaptation LoRA method:

[0068] Freezing the base model weights, reducing the computational resource requirements,

[0069] Setting fine-tuning parameters: determining learning rate, batch size, training rounds,

[0070] Effect verification and testing: regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust training strategies or stop training according to validation results;

[0071] Step 22: Adjust the TinyLlama LLM model using the retrieval augmented generation RAG method:

[0072] Preprocessing and text vectorization of the tuning dataset, mapping the tuning dataset text to a numerical vector space for subsequent similarity calculations,

[0073] Create an index and perform query conversion: convert user input questions or instructions into vector form, perform approximate nearest neighbor ANN search in the vector database, find the most similar document vectors to the query vector, and use them to complete the TinyLlama LLM model retrieval and generated sentence adjustment.

[0074] The information interaction and execution process between the modules of the above device are based on the same concept as the method embodiments of the present application, and the specific content can be referred to the description in the method embodiments of the present application, which will not be repeated here.

[0075] Similarly, the device of the present application realizes model tuning through the low-rank adaptation (LoRA) and retrieval augmented generation (RAG) methods in the field of data element circulation, greatly improving the accuracy of the TinyLlama model in converting natural language to SQL in the field of data elements. Users can further quickly and accurately query the data they want through natural language, improving the flexibility and ease of use of the system and reducing the difficulty of using the system.

[0076] It should be noted that not all steps and modules in the above processes and device structures are necessary, and some steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in each of the above embodiments can be a physical structure or a logical structure, that is, some modules can be implemented by the same physical entity, or some modules can be implemented by multiple physical entities, or can be implemented by some components in multiple independent devices together.

[0077] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any equivalent replacement or transformation made by those skilled in the art based on the present application shall fall within the protection scope of the present application. The protection scope of the present application is defined by the claims.

Claims

1. A method for fine-tuning TinyLlama LLM, characterized by include: Step 1: Develop the required tuning dataset for the TinyLlama LLM model: Step 11: When using data elements, record each data call information through logs. The data call information includes the called model, method, and executed SQL. Use log data as the tuning data set. When using data elements, perform data conversion, including: in the data elements, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship. Record the mapping relationship so that data conversion can be completed directly during data extraction. Step 12: Import the b-mc2 / sql-create-context dataset from the third-party data source HuggingFace to supplement and optimize the dataset, including: Translate the question column of the b-mc2 / sql-create-context dataset into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand. Step 2: Fine-tune the TinyLlama LLM model using the tuning dataset: Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method: Freeze the basic model weights to reduce computing resource requirements, Set fine-tuning parameters: determine the learning rate, batch size, and number of training rounds, Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results; Step 22: Tune the TinyLlama LLM model using the retrieval-augmented RAG generation method: Perform tuning dataset preprocessing and text vectorization, mapping the tuning dataset text to a numerical vector space to facilitate subsequent similarity calculations. Create an index and perform query conversion: Convert the user's input questions or instructions into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector most similar to the query vector to complete the adjustment of TinyLlama LLM model retrieval and generated sentences.

2. A method for fine-tuning TinyLlama LLM according to claim 1, characterized in that In step 11, Elasticsearch Logstash is used to extract log data, temporarily store the extracted data in the front-end database, and perform data cleaning, including: data cleaning of the data in the front-end database. Data cleaning includes removing information, correcting spelling errors, and filtering out low-quality data.

3. A device for fine-tuning TinyLlama LLM, characterized by Including data set management module and tuning module, The dataset management module develops the tuning dataset required for the TinyLlama LLM model and performs the following steps: Step 11: When using data elements, log each data call information, which includes the called model, method, and executed SQL. Use log data as the tuning data set. The data set management module performs data conversion when using data elements, including: in the data elements, the model represents the table or view in the database, the attribute represents the field in the database, and the attribute type also has a corresponding mapping relationship. Record the mapping relationship so that data conversion can be completed directly during data extraction. Step 12: Import the b-mc2 / sql-create-context dataset from the third-party data source HuggingFace to supplement and optimize the dataset, including: Translate the question column of the b-mc2 / sql-create-context dataset into Chinese data, use the Chinese data to supplement and tune the dataset, and define a chat template to convert the Chinese data into a format that TinyLlama can understand. The tuning module fine-tunes the TinyLlama LLM model using the tuning dataset, performing the following steps: Step 21: Fine-tune the TinyLlama LLM model using the Low Rank Adaptation LoRA method: Freeze the basic model weights to reduce computing resource requirements, Set fine-tuning parameters: determine the learning rate, batch size, and number of training rounds, Perform effect verification and testing: Regularly evaluate model performance on the validation set, monitor overfitting and performance indicators, and adjust the training strategy or stop training based on the verification results; Step 22: Tune the TinyLlama LLM model using the retrieval-augmented RAG generation method: Perform tuning dataset preprocessing and text vectorization, mapping the tuning dataset text to a numerical vector space to facilitate subsequent similarity calculations. Create an index and perform query conversion: Convert the user's input questions or instructions into vector form, perform an approximate nearest neighbor ANN search in the vector database, and find the document vector most similar to the query vector to complete the adjustment of TinyLlama LLM model retrieval and generated sentences.

4. A device for fine-tuning TinyLlama LLM according to claim 3, characterized in that the data When the set management module executes step 11, it uses Elasticsearch Logstash to extract log data, temporarily stores the extracted data in the front-end database, and performs data cleaning, including: cleaning the data in the front-end database. Data cleaning includes removing information, correcting spelling errors, and filtering out low-quality data.

Images