Large language model fine tuning method and system and computer storage medium
By constructing a sample database of multiple police case cases and performing low-rank adaptation fine-tuning, the problem of inaccurate recommendation in large-scale case cases is solved, and higher police case cases recommendation accuracy and model performance are achieved.
Patent Information
- Application Number
- CN202510019519.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-05-06
AI Technical Summary
In the task of recommending police incident cases, the traditional large-language model fine-tuning method leads to inaccuracy in the recommendation of police incident cases when facing a large amount of case data.
By constructing a sample database covering multiple cases of alarm cases, the samples were processed to form a verification data set, the training model was fine-tuned twice by using the low-rank adaptive fine-tuning method, and the fine-tuning results were combined for quantification, to obtain the first fine-tuning model. The model saves its weight after verifying that the preset conditions are met on the verification dataset.
It significantly improves the accuracy of police case recommendations, while reducing the training cost and time of large language models, and improving the performance of the model.
Smart Images
Figure CN119938844A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of model fine-tuning, and further to a large language model fine-tuning method, system and computer storage medium. Background Art
[0002] With the rapid development of artificial intelligence technology, large language models have proven their outstanding capabilities in multiple fields such as natural language processing and computer vision. However, when these large models are directly applied to the task of recommending police case causes, they usually need to be fine-tuned specifically for the characteristics of the case causes in order to better realize their potential. Although traditional fine-tuning methods are effective to a certain extent, when faced with situations such as large amounts of case cause data and a wide range of choices, large language models fine-tuned by traditional fine-tuning methods often lead to inaccurate police case cause recommendations. Summary of the invention
[0003] In order to solve the above technical problems, the present application provides a large language model fine-tuning method, system and computer storage medium, which improves the accuracy of police case cause recommendation.
[0004] In a first aspect, the present application provides a large language model fine-tuning method, comprising: establishing a sample database covering a variety of police case causes based on collected police data; processing each sample in the sample database to obtain a verification data set; performing low-rank adaptive fine-tuning on the training model based on the multiple police case causes and the police descriptions in the sample database; merging the results of two low-rank adaptive fine-tunings and performing model quantization to obtain a first fine-tuning model; verifying the verification data set based on the first fine-tuning model, and saving the weight of the first fine-tuning model when the verification result meets the preset conditions.
[0005] The large language model fine-tuning method constructs a sample database containing a variety of police case causes, processes the samples to form a verification data set, performs two low-rank adaptive fine-tuning on different police case causes and police descriptions, and merges and quantifies the two fine-tuning results to obtain a first fine-tuning model. By verifying the first fine-tuning model on the verification data set and saving the model weights after the verification results meet the preset conditions, the first fine-tuning model can significantly improve the accuracy of police case cause recommendations during subsequent deployment and use.
[0006] In one implementation, the low-rank adaptive fine-tuning of the model to be trained is performed based on the multiple police case causes and the police descriptions in the sample database, specifically including: converting the format of the police description and the multiple police case causes respectively through a data preprocessing function, and using the police description after format conversion as the first training sample, and using the multiple police case causes after format conversion as the second training sample; configuring the low-rank adaptive fine-tuning parameters, the fine-tuning parameters include the low-rank adaptive hyperparameters, the fine-tuned model layer, rank and dropout ratio; configuring the training parameters in the fine-tuning process, the training parameters include the learning rate, batch size and training rounds; based on the first training sample, the fine-tuning parameters and the training parameters, the initialized model to be trained is subjected to low-rank adaptive fine-tuning to obtain a second fine-tuned model; based on the second training sample, the fine-tuning parameters and the training parameters, the initialized model to be trained is subjected to low-rank adaptive fine-tuning to obtain a third fine-tuned model.
[0007] The large language model fine-tuning method can optimize the model structure, reduce the risk of overfitting, and improve the generalization ability of the model by configuring the fine-tuning parameters of low-rank adaptation, including hyperparameters, model layers, ranks, and dropout ratios. At the same time, by setting reasonable training parameters, such as learning rate, batch size, and training rounds, the convergence speed of the model can be accelerated and the stability of the training process can be improved. Finally, by performing two low-rank adaptive fine-tuning on the training model, respectively for the police description and the police case, the second fine-tuning model and the third fine-tuning model can be obtained. The first fine-tuning model after merging and quantizing the second fine-tuning model and the third fine-tuning model is verified, and the weight of the first fine-tuning model is saved when the verification result meets the preset conditions. The first fine-tuning model can significantly improve the accuracy of the police case recommendation in the subsequent deployment and use, while reducing the training cost and time of the large language model, and improving the performance of the large language model.
[0008] In one implementation, the step of merging the results of two low-rank adaptive fine-tuning operations and performing model quantization to obtain a first fine-tuning model specifically includes: merging the second fine-tuning model and the third fine-tuning model, and fusing the weights of the second fine-tuning model with the weights of the third fine-tuning model; converting the weights of the merged model from floating point numbers to fixed point numbers, and using the model after weight conversion as the first fine-tuning model.
[0009] In one implementation, it also includes: when the verification result does not meet the preset conditions, adjusting the parameters of the model to be trained; and repeatedly executing the step of verifying the verification data set until the verification result meets the preset conditions.
[0010] In one implementation, it also includes: processing each sample in the sample database to obtain a fine-tuning data set; and based on the fine-tuning data set, performing performance evaluation on the second fine-tuning model and the third fine-tuning model respectively.
[0011] In one implementation, it also includes: calculating performance indicators based on the first fine-tuning model and the verification data set, and the performance indicators include accuracy, precision, recall rate, and F1 score.
[0012] In the second aspect, the present application also provides a large language model fine-tuning system, including: an establishment module, configured to establish a sample database covering various police case causes based on the collected police data; a processing module, configured to process each sample in the sample database to obtain a verification data set; a model fine-tuning module, configured to perform low-rank adaptive fine-tuning on the training model based on the various police case causes and the police descriptions in the sample database; a merging module, configured to merge the results of two low-rank adaptive fine-tunings and perform model quantization to obtain a first fine-tuning model; a verification module, configured to verify the verification data set based on the first fine-tuning model, and save the weight of the first fine-tuning model when the verification result meets the preset conditions.
[0013] In one implementation, it also includes: a format conversion module, which is configured to perform format conversion on the police description and the multiple police case causes respectively through a data preprocessing function, and use the police description after format conversion as the first training sample, and use the multiple police case causes after format conversion as the second training sample; a parameter configuration module, which is configured to configure the fine-tuning parameters of the low-rank adaptation, and the fine-tuning parameters include the hyperparameters of the low-rank adaptation, the fine-tuned model layer, the rank and the dropout ratio; the parameter configuration module is also configured to configure the training parameters in the fine-tuning process, and the training parameters include the learning rate, the batch size and the training rounds; the model fine-tuning module is also configured to perform low-rank adaptive fine-tuning on the initialized model to be trained based on the first training sample, the fine-tuning parameters and the training parameters to obtain a second fine-tuning model; the model fine-tuning module is also configured to perform low-rank adaptive fine-tuning on the initialized model to be trained based on the second training sample, the fine-tuning parameters and the training parameters to obtain a third fine-tuning model.
[0014] In one implementation, the merging module is further configured to merge the second fine-tuning model and the third fine-tuning model, and to fuse the weights of the second fine-tuning model with the weights of the third fine-tuning model; the merging module is further configured to convert the weights of the merged model from floating point numbers to fixed point numbers, and to use the model after weight conversion as the first fine-tuning model.
[0015] In a third aspect, the present application also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above steps of the large language model fine-tuning method.
[0016] Compared with the prior art, the present invention has at least one of the following beneficial effects:
[0017] 1. By constructing a sample database containing a variety of police case causes and processing the samples to form a verification data set, two low-rank adaptive fine-tunings are performed for different police case causes and police case descriptions, and the two fine-tuning results are merged and quantified to obtain the first fine-tuning model. By verifying the first fine-tuning model on the verification data set and saving the model weights after the verification results meet the preset conditions, the first fine-tuning model can significantly improve the accuracy of police case cause recommendations during subsequent deployment and use.
[0018] 2. By configuring the fine-tuning parameters of low-rank adaptation, including hyperparameters, model layers, ranks, and dropout ratios, the model structure can be optimized, the risk of overfitting can be reduced, and the generalization ability of the model can be improved. At the same time, by setting reasonable training parameters, such as learning rate, batch size, and training rounds, the convergence speed of the model can be accelerated and the stability of the training process can be improved. Finally, by performing two low-rank adaptive fine-tuning on the training model, the second fine-tuning model and the third fine-tuning model can be obtained for the police description and the police case cause respectively. The first fine-tuning model after merging and quantizing the second fine-tuning model and the third fine-tuning model is verified, and the weight of the first fine-tuning model is saved when the verification result meets the preset conditions. The first fine-tuning model can significantly improve the accuracy of the police case cause recommendation during subsequent deployment and use, while reducing the training cost and time of the large language model, and improving the performance of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The preferred implementation modes will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present invention.
[0020] Figure 1 A flowchart of a large language model fine-tuning method provided in an embodiment of the present application is shown;
[0021] Figure 2 A flow chart of model fine-tuning provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.
[0023] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".
[0024] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0025] In this document, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0026] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0027] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.
[0028] Large Language Model (LLM) refers to a model that can process large amounts of text data and generate natural language text. Common large language models include OpenAI's GPT series, Google's BERT and T5, and Tongyi Qianwen Large Model (or Qwen Large Model). They play an important role in the field of natural language processing and are widely used in machine translation, text generation, dialogue systems, and police case recommendation.
[0029] In terms of recommending police case causes, the large language model needs to be fine-tuned according to the characteristics of the case causes in order to give full play to its potential, so that it can analyze and understand the semantic information of the alarm content, and recommend cases related to the police situation by extracting key information and contextual relationships. Among them, fine-tuning the large language model refers to further performing a small amount of supervised learning on the already trained large language model to adapt to specific natural language processing tasks. In the embodiment of the present application, the large language model is first fine-tuned based on the police case causes, so that the large language model has a qualitative definition of the police case causes, and then the large language model is fine-tuned with the actual police description, which can achieve at least one of the following beneficial effects: improving the accuracy of the large language model's recommendation of police case causes; or, reducing the training cost and time of the large language model, and improving the performance of the large language model.
[0030] The following is explained with reference to the accompanying drawings:
[0031] Reference Figure 1 , which shows a flow chart of a large language model fine-tuning method provided by an embodiment of the present application. Figure 1 As shown, including:
[0032] S100, based on the collected police data, establish a sample database covering various police case causes.
[0033] S110, processing each sample in the sample database to obtain a verification data set.
[0034] S120, based on various police case causes and police case descriptions in the sample database, low-rank adaptive fine-tuning is performed on the training model.
[0035] S130, combining the results of the two low-rank adaptive fine-tuning operations and performing model quantization to obtain a first fine-tuning model.
[0036] S140, verifying the verification data set based on the first fine-tuning model, and saving the weight of the first fine-tuning model when the verification result meets a preset condition.
[0037] Low-Rank Adaptation (LoRA) is a technique for fine-tuning large language models. The core idea of this technique is to introduce a low-rank matrix into the model's weight matrix to achieve fine-tuning of the model without changing all the parameters of the original pre-trained model.
[0038] Based on more than 300 types of police cases officially released, collect police data related to various police cases, such as call records, electronic transcripts, and case records. Classify and organize the collected police data to form a sample database covering different police cases, while ensuring that each sample in the sample database contains necessary metadata, such as police case, police description, etc. For police cases with a small number of samples, data enhancement techniques, such as synonym replacement, can be used to generate more samples. This can be achieved through natural language processing tools to improve the generalization ability of large language models.
[0039] Label each sample in the sample database to ensure that each sample can accurately reflect its police case. Perform manual analysis on each sample in the sample database, delete erroneous police cases and police case associations, and clarify the data in the sample database to remove irrelevant information. Then split the sample database into a fine-tuning data set and a verification data set for subsequent large language model training and evaluation. Write relevant explanatory data for each police case in the sample database, such as the definition of the police case, legal basis, possible legal consequences, and related processing procedures.
[0040] Install the necessary Python dependency library transformers to support the loading, data processing and fine-tuning of the large language model. According to the recommended requirements and computing resources for police cases, select the Qwen large model version Qwen2.5-7B / 72B as the model to be trained, and download the pre-trained file of the model to be trained. Configure high-performance computing resources to meet the running requirements of the model to be trained. First, based on the explanation data related to various police cases, the low-rank adaptive fine-tuning of the model to be trained is performed to allow the model to have a qualitative definition of the police case, and then the police description in the actual sample database is used to perform low-rank adaptive fine-tuning on the large language model. Combine the fine-tuning results for various police cases and the fine-tuning results for police descriptions, and quantify the merged model through llama.cpp to obtain the first fine-tuning model. Load the first fine-tuning model into the validation data set for verification, and analyze the performance of the first fine-tuning model on the validation data set. If the verification result meets the preset conditions, it means that the first fine-tuning model performs well on the validation data set, and then save the weight of the first fine-tuning model for subsequent deployment and use. Before validation, you need to ensure the independence of the validation dataset, that is, the validation dataset is independent of the fine-tuning dataset and has not been used in the fine-tuning process. The validation dataset represents the real environment where the first fine-tuned model will be deployed.
[0041] The embodiment of the present application constructs a sample database containing a variety of police case causes, processes the samples to form a verification data set, performs two low-rank adaptive fine-tuning on different police case causes and police descriptions, and merges and quantifies the two fine-tuning results to obtain a first fine-tuning model. By verifying the first fine-tuning model on the verification data set and saving the model weights after the verification results meet the preset conditions, the first fine-tuning model can significantly improve the accuracy of the police case cause recommendation during subsequent deployment and use.
[0042] Reference Figure 2 , which shows a flow chart of a model fine-tuning provided by an embodiment of the present application. Figure 2 As shown, including:
[0043] S200, respectively converting the formats of the police situation description and the various police situation causes through a data preprocessing function, and using the police situation description after the format conversion as a first training sample, and using the various police situation causes after the format conversion as a second training sample.
[0044] S210, configuring the fine-tuning parameters of low-rank adaptation, the fine-tuning parameters including the hyperparameters of low-rank adaptation, the fine-tuned model layer, the rank, and the dropout ratio.
[0045] S220, configuring training parameters in the fine-tuning process, where the training parameters include a learning rate, a batch size, and a training round.
[0046] S230: Based on the first training sample, the fine-tuning parameters, and the training parameters, perform low-rank adaptive fine-tuning on the initialized model to be trained to obtain a second fine-tuning model.
[0047] S240: Based on the second training sample, the fine-tuning parameters, and the training parameters, perform low-rank adaptive fine-tuning on the initialized model to be trained to obtain a third fine-tuning model.
[0048] When the training model is fine-tuned for low-rank adaptation through various police cases, first write a data preprocessing function. The data preprocessing function converts the explanation data related to various police cases into a format suitable for model input by connecting to the sample database after manual analysis, such as setting prompt words, word segmentation, adding special tags (tokens) and other processing for the explanation data related to various police cases. Among them, the model to be trained and the corresponding word segmenter are loaded through the AutoModelForCausalLM class and AutoTokenizer class in the transformers library. The explanation data of various police cases after format conversion are used as the second training sample, and the weight of the model to be trained is loaded to initialize the model to be trained. The parameters of low-rank adaptation are configured, for example, the fine-tuned model layer is the attention mechanism layer in the fine-tuning Transformer architecture, the rank of low-rank adaptation is set to 16, the hyperparameter (alpha) of low-rank adaptation is set to 32, and the dropout ratio is set to 0.2, where the dropout ratio can also be called the random inactivation ratio. Configure the training parameters in the fine-tuning process, such as the learning rate is 0.032, the batch size is 64, and the training rounds are 4 rounds. Furthermore, based on the second training sample and the configured fine-tuning parameters and training parameters, the initialized model to be trained is subjected to low-rank adaptive fine-tuning to obtain a third fine-tuning model.
[0049] After the low-rank adaptation fine-tuning of the training model through a variety of police case causes, the low-rank adaptation fine-tuning of the training model is performed on the police description. Similarly, the data preprocessing function converts the police description in the sample database into a format suitable for model input by connecting the sample database after manual analysis, such as setting prompt words, word segmentation, adding special tags (tokens) and other processing for the police description. The police description after format conversion is used as the first training sample, and the weight of the model to be trained is loaded to initialize the model to be trained. The parameters of low-rank adaptation are configured, for example, the fine-tuned model layer is the attention mechanism layer in the fine-tuned Transformer architecture, the rank of low-rank adaptation is set to 16, the hyperparameter (alpha) of low-rank adaptation is set to 32, and the dropout ratio is set to 0.2. The training parameters in the fine-tuning process are configured, such as the learning rate is 0.032, the batch size is 64, and the training rounds are 4 rounds. Further, based on the first training sample and the configured fine-tuning parameters and training parameters, the initialized model to be trained is low-rank adapted and fine-tuned to obtain the second fine-tuning model.
[0050] Furthermore, the second fine-tuning model and the third fine-tuning model are merged, and the merged model is quantized through llama.cpp to obtain the first fine-tuning model. The first fine-tuning model is loaded into the validation data set for verification, and the performance of the first fine-tuning model on the validation data set is analyzed. If the verification result meets the preset conditions, it means that the first fine-tuning model performs well on the validation data set, and then the weight of the first fine-tuning model is saved for subsequent deployment and use.
[0051] The embodiment of the present application can optimize the model structure, reduce the risk of overfitting, and improve the generalization ability of the model by configuring the fine-tuning parameters of low-rank adaptation, including hyperparameters, model layers, ranks, and dropout ratios. At the same time, by setting reasonable training parameters, such as learning rate, batch size, and training rounds, the convergence speed of the model can be accelerated and the stability of the training process can be improved. Finally, by performing two low-rank adaptive fine-tuning on the training model, respectively for the police description and the police case, a second fine-tuning model and a third fine-tuning model can be obtained. The first fine-tuning model after merging and quantizing the second fine-tuning model and the third fine-tuning model is verified, and the weight of the first fine-tuning model is saved when the verification result meets the preset conditions. The first fine-tuning model can significantly improve the accuracy of the recommendation of the police case in the subsequent deployment and use, while reducing the training cost and time of the large language model, and improving the performance of the large language model.
[0052] In one embodiment of the present application, the results of two low-rank adaptive fine-tuning are merged and the model is quantized to obtain a first fine-tuning model, specifically including: merging the second fine-tuning model and the third fine-tuning model, and fusing the weights of the second fine-tuning model with the weights of the third fine-tuning model; converting the weights of the merged model from floating point numbers to fixed point numbers, and using the model after weight conversion as the first fine-tuning model.
[0053] Through the explanation data related to a variety of police cases and the police descriptions of the sample database, the training model is respectively subjected to low-rank adaptive fine-tuning, and the second fine-tuning model and the third fine-tuning model are obtained. By loading the second fine-tuning model and the third fine-tuning model, and merging their weights according to a simple weight averaging strategy. And the weights of the merged model are converted from floating point numbers to fixed point numbers, thereby reducing the size of the model. The model after weight conversion is further used as the first fine-tuning model, and the first fine-tuning model is loaded into the verification data set for verification, and the performance of the first fine-tuning model on the verification data set is analyzed. If the verification result meets the preset conditions, the weight of the first fine-tuning model is saved for subsequent deployment and use. In the embodiment of the present application, the model to be trained, the first fine-tuning model, the second fine-tuning model, and the third fine-tuning model are all Qwen large models (of course, other large language models can also be used), and llama.cpp can be used to implement the format conversion, quantization, reasoning, and deployment of the Qwen large model.
[0054] In one embodiment of the present application, it also includes: when the verification result does not meet the preset conditions, adjusting the parameters of the model to be trained; repeating the step of verifying the verification data set until the verification result meets the preset conditions.
[0055] When the verification result does not meet the preset conditions, it means that the first fine-tuning model performs poorly on the verification data set, and it may be necessary to return to the step of low-rank adaptive fine-tuning to adjust the model to be trained or the training strategy. According to the verification results, make necessary adjustments to the model to be trained, such as but not limited to changing the structure of the model to be trained, adjusting hyperparameters, or further fine-tuning. After adjusting the model to be trained, repeat the steps of verifying the verification data set until the verification result meets the preset conditions. At the same time, regardless of whether the verification result meets the preset conditions, all key information in the verification process is recorded, such as but not limited to model configuration, performance indicators, and any adjustment-related content.
[0056] In one embodiment of the present application, the method further includes: processing each sample in the sample database to obtain a fine-tuning data set; and performing performance evaluation on the second fine-tuning model and the third fine-tuning model based on the fine-tuning data set.
[0057] In the embodiment of the present application, after obtaining the second fine-tuning model and the third fine-tuning model, the performance of the second fine-tuning model and the third fine-tuning model can be evaluated respectively through the fine-tuning data set, so as to ensure that the second fine-tuning model can accurately classify the police situation description, and ensure that the third fine-tuning model can accurately classify various police situation cases.
[0058] In one embodiment of the present application, it also includes: calculating performance indicators based on the first fine-tuning model and the verification data set, and the performance indicators include accuracy, precision, recall rate, and F1 score.
[0059] The embodiment of the present application also provides a large language model fine-tuning system, including: an establishment module, configured to establish a sample database covering various police case causes based on the collected police data; a processing module, configured to process each sample in the sample database to obtain a verification data set; a model fine-tuning module, configured to perform low-rank adaptive fine-tuning on the training model based on various police case causes and police descriptions in the sample database; a merging module, configured to merge the results of two low-rank adaptive fine-tunings and perform model quantization to obtain a first fine-tuning model; a verification module, configured to verify the verification data set based on the first fine-tuning model, and save the weight of the first fine-tuning model when the verification result meets the preset conditions.
[0060] The details of the embodiments of the present application have been described in the foregoing embodiments and will not be repeated here.
[0061] In one embodiment of the present application, it also includes: a format conversion module, which is configured to perform format conversion on the police description and various police case causes respectively through a data preprocessing function, and use the police description after format conversion as the first training sample, and use the various police case causes after format conversion as the second training sample; a parameter configuration module, which is configured to configure the fine-tuning parameters of low-rank adaptation, and the fine-tuning parameters include the hyperparameters of low-rank adaptation, the fine-tuned model layer, the rank and the dropout ratio; the parameter configuration module is also configured to configure the training parameters in the fine-tuning process, and the training parameters include the learning rate, the batch size and the training rounds; the model fine-tuning module is also configured to perform low-rank adaptive fine-tuning on the initialized model to be trained based on the first training sample, the fine-tuning parameters and the training parameters to obtain the second fine-tuning model; the model fine-tuning module is also configured to perform low-rank adaptive fine-tuning on the initialized model to be trained based on the second training sample, the fine-tuning parameters and the training parameters to obtain the third fine-tuning model.
[0062] In one embodiment of the present application, the merging module is further configured to merge the second fine-tuning model and the third fine-tuning model, and to fuse the weights of the second fine-tuning model with the weights of the third fine-tuning model; the merging module is further configured to convert the weights of the merged model from floating point numbers to fixed point numbers, and to use the model after weight conversion as the first fine-tuning model.
[0063] An embodiment of the present application also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the large language model fine-tuning method described in any of the above embodiments are implemented.
[0064] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.
Claims
1. A large language model fine-tuning method, characterized in that: include: Based on the collected police data, a sample database covering various police cases is established; Processing each sample in the sample database to obtain a verification data set; Based on the multiple police case causes and police descriptions in the sample database, respectively performing low-rank adaptive fine-tuning on the training model; The results of the two low-rank adaptive fine-tuning are combined and the model is quantized to obtain the first fine-tuning model; The verification data set is verified based on the first fine-tuning model, and when the verification result meets a preset condition, the weight of the first fine-tuning model is saved.
2. The large language model fine-tuning method according to claim 1, characterized in that: Based on the multiple police case causes and police descriptions in the sample database, the training model is subjected to low-rank adaptive fine-tuning, specifically including: The police situation description and the multiple police situation causes are formatted respectively by a data preprocessing function, and the police situation description after format conversion is used as a first training sample, and the multiple police situation causes after format conversion are used as a second training sample; Configuring fine-tuning parameters of the low-rank adaptation, wherein the fine-tuning parameters include hyperparameters of the low-rank adaptation, a fine-tuned model layer, a rank, and a dropout ratio; Configure training parameters during fine-tuning, including learning rate, batch size, and training rounds; Based on the first training sample, the fine-tuning parameter, and the training parameter, low-rank adaptive fine-tuning is performed on the initialized model to be trained to obtain a second fine-tuning model; Based on the second training sample, the fine-tuning parameters, and the training parameters, low-rank adaptive fine-tuning is performed on the initialized model to be trained to obtain a third fine-tuning model.
3. The large language model fine-tuning method according to claim 2, characterized in that: The step of combining the results of two low-rank adaptive fine-tuning operations and performing model quantization to obtain a first fine-tuning model specifically includes: Merging the second fine-tuning model and the third fine-tuning model, and fusing the weight of the second fine-tuning model with the weight of the third fine-tuning model; The weights of the merged model are converted from floating point numbers to fixed point numbers, and the model after the weight conversion is used as the first fine-tuning model.
4. The large language model fine-tuning method according to claim 1, characterized in that: Also includes: When the verification result does not meet the preset conditions, adjusting the parameters of the model to be trained; Repeat the step of verifying the verification data set until the verification result meets a preset condition.
5. The large language model fine-tuning method according to claim 2, characterized in that: Also includes: Processing each sample in the sample database to obtain a fine-tuning data set; Based on the fine-tuning dataset, performance evaluation is performed on the second fine-tuning model and the third fine-tuning model respectively.
6. The large language model fine-tuning method according to any one of claims 1 to 5, characterized in that: Also includes: Based on the first fine-tuning model and the validation data set, performance indicators are calculated, and the performance indicators include accuracy, precision, recall, and F1 score.
7. A large language model fine-tuning system, characterized in that: include: A building module is configured to build a sample database covering various police case causes based on the collected police data; A processing module, configured to process each sample in the sample database to obtain a verification data set; A model fine-tuning module is configured to perform low-rank adaptive fine-tuning on the training model based on the multiple police case causes and police descriptions in the sample database; A merging module is configured to merge the results of two low-rank adaptive fine-tuning and perform model quantization to obtain a first fine-tuning model; The verification module is configured to verify the verification data set based on the first fine-tuning model, and when the verification result meets a preset condition, save the weight of the first fine-tuning model.
8. The large language model fine-tuning system according to claim 7, characterized in that: Also includes: A format conversion module is configured to perform format conversion on the police situation description and the multiple police situation causes respectively through a data preprocessing function, and use the police situation description after format conversion as a first training sample, and use the multiple police situation causes after format conversion as a second training sample; A parameter configuration module, configured to configure fine-tuning parameters of the low-rank adaptation, wherein the fine-tuning parameters include hyperparameters of the low-rank adaptation, a fine-tuned model layer, a rank, and a dropout ratio; The parameter configuration module is further configured to configure training parameters in the fine-tuning process, wherein the training parameters include a learning rate, a batch size, and a training round; The model fine-tuning module is further configured to perform low-rank adaptive fine-tuning on the initialized model to be trained based on the first training sample, the fine-tuning parameter, and the training parameter to obtain a second fine-tuning model; The model fine-tuning module is further configured to perform low-rank adaptive fine-tuning on the initialized model to be trained based on the second training sample, the fine-tuning parameters, and the training parameters to obtain a third fine-tuning model.
9. The large language model fine-tuning system according to claim 8, characterized in that: The merging module is further configured to merge the second fine-tuning model and the third fine-tuning model, and fuse the weight of the second fine-tuning model with the weight of the third fine-tuning model; The merging module is also configured to convert the weights of the merged model from floating point numbers to fixed point numbers, and use the model after the weight conversion as the first fine-tuning model.
10. A computer storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the large language model fine-tuning method described in any one of claims 1 to 6 are implemented.