Large language model fine tuning system

Abstract

The invention discloses a large language model fine tuning system, and relates to the technical field of large language models, and the system comprises a model loading module which is used for loading a pre-adjusted large language model; the fine adjustment module comprises a fine adjustment unit and an interpretation unit, and the fine adjustment unit is used for receiving the processed data transmitted by the preprocessing module, training to obtain adjustment parameters of the large language model of the model loading module and adjusting the large language model of the model loading module according to the adjustment parameters; the fine tuning unit receives the preprocessed data and performs fine tuning on the model, and the interpretation unit outputs the decision process of the model, so that medical professionals can understand the internal logic of the model, the adaptability and accuracy of the model are improved, and the correction module receives the decision process of the interpretation unit and generates evaluation indexes according to the decision process. And outputting the corrected large language model. The fairness and ethics of medical decision making are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and more specifically, to a large language model fine-tuning system. Background Art

[0002] With the rapid development of artificial intelligence technology, although large language models perform well in processing general language tasks, their application in specific fields, especially in the medical field, still has a lot of room for improvement. The communication, recording and decision-making processes in hospitals are full of professional terms and complex medical information, so hospitals need to fine-tune the use of large language models in a targeted manner; However, in actual use, the decision-making process of large language models in the medical field is often opaque, which is called the "black box" problem. In the medical field, the consequences of decisions may be life-threatening, so it is crucial to understand the logic behind the output generated by AI. Due to the complexity of medical large language models and the characteristics of deep learning algorithms, their internal working mechanisms are difficult for humans to understand and explain, which limits the application of models in clinical decision-making, because medical professionals need to trust the decision-making process of the model before they can use it with confidence; When used in medical settings, large language models may generate responses based solely on probabilistic associations between words, lacking true understanding and amplifying existing biases in the training data, particularly those related to demographics, disease prevalence, and treatment outcomes, which may lead to unfair and unequal medical outcomes. Summary of the invention

[0003] To solve the above problems, the present invention provides a large language model fine-tuning system.

[0004] The present invention provides a large language model fine-tuning system, comprising a model loading module, wherein the model loading module is used to load a pre-adjusted large language model; A preprocessing module, which is used to process the original medical text data and transmit the processed data; A fine-tuning module, the fine-tuning module comprising a fine-tuning unit and an interpretation unit, the fine-tuning unit being used to receive the processed data transmitted by the preprocessing module, train to obtain adjustment parameters of the large language model of the model loading module, and adjust the large language model of the model loading module according to the adjustment parameters to obtain a fine-tuned large language model; The interpretation unit is used to output the decision process of the fine-tuned large language model and transmit it; A correction module is used to correct the output result of the fine-tuned large language model and transmit the correction result as an auxiliary decision.

[0005] Preferably, the specific working steps of the model loading module are as follows: Identify the open source large language models to use; Prepare the Python environment and install the necessary package manager using pip; Download the required model files from the open source platform, including large language model weights and configuration files; Use a deep learning framework to load large language model weights; Set large language model parameters, including learning rate, optimizer selection, and number of fine-tuning layers; Get the pre-tuned large language model.

[0006] Preferably, the specific working steps of the preprocessing module are as follows: Obtaining original medical text data and classifying the original medical text data, specifically including electronic medical records, medical imaging reports, medical question and answer data, and clinical data; Then, according to the classification results of the original medical text data, each type of original medical text data is preprocessed, and the processed data is transmitted to the corresponding micro-carving module.

[0007] Preferably, the specific steps of preprocessing each type of original medical text data according to the classification results of the original medical text data are as follows: For the original medical text data of electronic medical records, we first remove the data of headers, footers, and template tags, and then standardize the date, time, and measurement units. We use named entity recognition technology to identify patient information and diagnosis results in the medical records, and use the mean or median to fill in missing values ​​to complete the preprocessing. For the original medical text data of medical imaging reports, first, descriptive text is extracted from the imaging report, the diagnostic conclusion is extracted as key information, and the extracted text is vectorized and converted into vector form to complete the preprocessing; For the original medical text data of the medical question and answer data type, first separate the questions and answers from the question and answer pairs, and standardize the questions and answers to complete the preprocessing; For raw medical text data of the clinical data type, clinical trial data from different sources are integrated into one dataset, and the data are converted into a unified format to complete preprocessing.

[0008] Preferably, the specific working steps of the fine-tuning unit are as follows: Receiving the processed data transmitted by the preprocessing module, and dividing the processed data into a training set X and a verification set Y; Load a large pre-trained language model and set key hyperparameters during fine-tuning, including learning rate, batch size, and number of training epochs; Use the preprocessed data to train the model and update the weights of the large language model. The specific calculation steps are: During the training process, the model is evaluated regularly using the validation set, and early stopping parameters are set, including the minimum change to be monitored, the maximum number of iterations allowed, and performance metrics. When the performance on the validation set no longer improves, training is stopped to obtain a fine-tuned large language model.

[0009] Preferably, the specific working steps of the interpretation unit include: Receiving the fine-tuned large language model and pre-input feature samples, inputting the pre-input feature samples into the large language model, and generating a prediction result of the large language model; The number of layers n in the fine-tuned large language model in the process of generating prediction results; Each feature in the pre-input feature sample in the large language model is labeled as , Indicates Features Get the weight corresponding to each layer of the large language model and attention score , weight and attention score Expressed as The weights and Attention scores; Multiply the weight of each layer by the attention score to get the feature Contributions of importance; Add up the contributions of all layers to get the feature Total importance score , Indicates The total importance score of the features; All features Total importance score As the decision process of the fine-tuned large language model, and transfer.

[0010] Preferably, the weight corresponding to each layer of the large language model is obtained and attention score The specific steps are as follows: For each layer, the corresponding weight , determine the layer index i to be queried in the model parameter list to locate the weight parameters of the layer; extract and record the weight parameter values ​​of the layer, which are learned during the model training process; During the forward propagation of the model, the query, key, and value are first calculated. The dot product of the query and the key is calculated to obtain the original attention score. The original attention score is scaled, usually divided by the square root of the key vector dimension, and the softmax function is applied to normalize the scaled attention score to obtain the final attention weight.

[0011] Preferably, the specific working steps of the interpretation unit also include: After the fine-tuned large language model generates a prediction result, all features Total importance score Arrange in descending order and take out the total importance score The maximum and minimum values ​​of are the features that have the greatest and least impact on model prediction. The sample data after removing the features is input into the fine-tuned model again to obtain new output results. The new output result is also used as the decision process of the fine-tuned large language model and transmitted.

[0012] Preferably, the specific working steps of the correction module are as follows: The model output results and output changes after removing features are generated based on the output changes to show which feature changes will affect the model's prediction results. The counterfactual explanation is also output as the decision process of the fine-tuned large language model for reference by staff.

[0013] Beneficial effects: Through the fine-tuning module, the fine-tuning unit receives the preprocessed data and fine-tunes the model, while the interpretation unit outputs the decision-making process of the model, so that medical professionals can understand the internal logic of the model. It not only improves the adaptability and accuracy of the model, but also enhances the trust of medical professionals in the AI ​​system, ensuring the rationality and safety of clinical decisions. At the same time, the correction module receives the decision-making process of the interpretation unit, generates evaluation indicators based on it, and outputs the corrected large language model, which effectively reduces model bias and improves the fairness and ethics of medical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0015] In actual use, the decision-making process of large language models in the medical field is often opaque, which is called the "black box" problem. In the medical field, the consequences of decisions may be life-threatening, so it is crucial to understand the logic behind the output generated by AI. Due to the complexity of medical large language models and the characteristics of deep learning algorithms, their internal working mechanisms are difficult for humans to understand and explain, which limits the application of models in clinical decision-making, because medical professionals need to trust the decision-making process of the model before they can use it with confidence; When used in medical settings, large language models may generate responses based solely on probabilistic associations between words, lacking true understanding and amplifying existing biases in the training data, particularly those related to demographics, disease prevalence, and treatment outcomes, which may lead to unfair and unequal medical outcomes.

[0016] like Figure 1 As shown: A large language model fine-tuning system includes a model loading module, wherein the model loading module is used to load a pre-adjusted large language model; it should be noted that the large language model is an open source model on the market, and the pre-loading module will contain the initialization code of the model to ensure that the model can be loaded from a pre-trained state and is ready for fine-tuning; A preprocessing module, which is used to process the raw medical text data and transmit the processed data; it should be noted that the preprocessing module receives the raw medical text data, including medical records and medical reports, and transmits the cleaned, annotated and formatted training and validation data sets, which are ready to be input into the model for fine-tuning; A fine-tuning module, the fine-tuning module comprising a fine-tuning unit and an interpretation unit, the fine-tuning unit being used to receive the processed data transmitted by the preprocessing module, train to obtain adjustment parameters of the large language model of the model loading module, and adjust the large language model of the model loading module according to the adjustment parameters to obtain a fine-tuned large language model; The interpretation unit is used to output the decision process of the fine-tuned large language model and transmit it; A correction module is used to correct the output result of the fine-tuned large language model and transmit the correction result as an auxiliary decision.

[0017] As an optional embodiment: the specific working steps of the model loading module are as follows: Decide on the open source large language model to use; for example, DistilBERT or Roberta, which can be obtained from platforms such as HuggingFace; Prepare the Python environment and use pip to install the necessary package manager; if GPU acceleration is required, you need to install the corresponding version of CUDA; Download the required model files from the open source platform, including large language model weights and configuration files; Use a deep learning framework to load a large language model weight; in PyTorch, you can create a model instance through BertModel.from_pretrained or similar methods Set large language model parameters, including learning rate, optimizer selection, and number of fine-tuning layers; Get the pre-tuned large language model.

[0018] As an optional embodiment: the specific working steps of the preprocessing module are as follows: The original medical text data is obtained and classified into electronic medical records, medical imaging reports, medical question and answer data and clinical data. It should be noted that the electronic medical records include the patient's medical history, diagnosis results, treatment plan, etc. Medical imaging reports include radiology reports, pathology reports, etc., including image descriptions and diagnosis conclusions. Medical question and answer data sets include question and answer pairs between patients and doctors, which are used to build question and answer systems. Clinical trial data include the design, execution and results of clinical trials. Then, according to the classification results of the original medical text data, each type of original medical text data is preprocessed, and the processed data is transmitted to the corresponding micro-carving module.

[0019] It should be noted that given the diverse sources of medical data, including electronic medical records, medical imaging reports, medical question-and-answer data, and clinical trial data, each type of data has its own unique structure and content characteristics, and therefore needs to be classified and processed to adapt to its characteristics. Different types of data require different preprocessing measures. Through classification, the preprocessing process can be customized for each type of data, thereby improving the efficiency of preprocessing.

[0020] As an optional embodiment: the specific steps of preprocessing each type of original medical text data according to the classification results of the original medical text data are as follows: For the original medical text data of electronic medical records, first remove the data of headers, footers, and template tags, then standardize the date, time, and measurement units, use named entity recognition technology to identify patient information and diagnosis results in medical records, and use the mean or median to fill missing values ​​to complete preprocessing; NER = NER_model (EMR_text); For example, an electronic medical record contains a patient's medical history, diagnosis, and treatment plan. During preprocessing, first remove the header, footer, and template tags in the document, such as the footer information after "Patient Name: Zhang San". Then, unify all date and time formats into the "YYYY-MM-DD" format, and convert measurement units such as "5 feet" to "152.4 centimeters". Use NER technology to identify "diabetes" as a diagnosis, and fill in the missing blood pressure values ​​with the mean of the blood pressure values ​​in the dataset; Through standardization, the consistency and comparability of data are improved, so that the model can more accurately analyze and predict the health status of patients. The application of named entity recognition technology improves the model's ability to identify key medical information, which is helpful for subsequent diagnosis and treatment decisions. Missing value filling reduces data incompleteness and avoids data bias during model training. For the original medical text data of medical imaging reports, first, descriptive text is extracted from the imaging report, the diagnostic conclusion is extracted as key information, and the extracted text is vectorized and converted into vector form to complete the preprocessing; For example, a radiology report contains image descriptions and diagnostic conclusions. During preprocessing, descriptive text is extracted from the report, such as "pulmonary CT shows multiple nodules". Then, keyword extraction technology is used to extract "multiple nodules" as key information, and vectorization technology is used to convert the text into a numerical form that the model can process; By extracting key information, the model can quickly focus on the most important part of the report, improving the efficiency and accuracy of diagnosis. Text vectorization allows unstructured text data to be processed by the model, providing the necessary input format for the training of deep learning models. For the original medical text data of the medical question and answer data type, first separate the questions and answers from the question and answer pairs, and standardize the questions and answers to complete the preprocessing; For example, suppose there is a question and answer: "Patient: I always have headaches recently, what should I do?" "Doctor: It may be caused by high blood pressure, I need to check it out." During preprocessing, the question and answer are separated and standardized, such as replacing "always" with "often" and expressing them in a unified language. Standardization helps reduce language diversity, allowing the model to better understand and answer patients’ questions. By separating questions and answers, the model can more accurately learn the patterns of doctor-patient communication and improve the response quality and relevance of the question-answering system. For raw medical text data of the clinical data type, clinical trial data from different sources are integrated into one dataset, and the data are converted into a unified format to complete preprocessing.

[0021] It should be noted that, assuming that there are clinical trial data from different research centers, these data are integrated into one dataset during preprocessing and converted into a unified XML format for easy analysis and comparison. Data integration enables comprehensive analysis of clinical trial results across research centers, increasing the breadth and depth of research.

[0022] As an optional embodiment: the specific working steps of the fine-tuning unit are as follows: Receive the processed data transmitted by the preprocessing module, and divide the processed data into a training set X and a validation set Y; it should be noted that in this implementation, the ratio of the training set X to the validation set Y is 80% and 20%. For example, if the task is disease classification, X may contain a text feature vector, and Y is a corresponding disease category label; Load a pre-trained large language model and set key hyperparameters in the fine-tuning process, including learning rate, batch size, and number of training epochs; Set key hyperparameters in the fine-tuning process, including learning rate (e.g. 1e-5), batch size (e.g. 32), and number of training epochs (e.g. 3-5 epochs); The model is trained using the preprocessed data and the weights of the large language model are updated. It should be noted that in each batch, the loss between the predicted and actual labels is calculated and the model weights are updated through backpropagation. During the training process, the model is evaluated regularly using the validation set, and early stopping parameters are set, including the minimum change to be monitored, the maximum number of iterations allowed, and performance metrics. When the performance on the validation set no longer improves, training is stopped to obtain a fine-tuned large language model.

[0023] It should be noted that during the training process, the model is regularly evaluated using the validation set, and early stopping parameters are set, including the minimum change to be monitored (e.g. 1e-4), the maximum number of iterations allowed (e.g. 10 Epochs), and performance metrics (e.g. accuracy). After each Epoch, the performance of the model on the validation set is evaluated and compared with the performance of the previous Epoch. If the performance on the validation set does not improve significantly within several consecutive Epochs, the early stopping method is triggered and training is stopped. After training, the model weights that perform best on the validation set are selected and applied to the entire dataset to obtain a fine-tuned large language model. If the accuracy of the fine-tuned model on the validation set reaches 85% in the disease classification task and there is no further improvement within three consecutive epochs, then this model will be saved and used for the final evaluation of the test set.

[0024] As an optional embodiment: the specific working steps of the interpretation unit include: Receiving the fine-tuned large language model and the pre-input feature samples, inputting the pre-input feature samples into the large language model, and generating a prediction result of the large language model; it should be noted that the pre-input feature samples are the basis for the model to make predictions, and they contain information required by the model to generate prediction results; The number of layers n in the fine-tuned large language model in the process of generating prediction results; Each feature in the pre-input feature sample in the large language model is labeled as , Indicates Features Get the weight corresponding to each layer of the large language model and attention score , weight and attention score Expressed as The weights and Attention scores; Multiply the weight of each layer by the attention score to get the feature Contributions of importance; Add up the contributions of all layers to get the feature Total importance score , Indicates The total importance score of the features; All features Total importance score As a decision-making process of the fine-tuned large language model, and then transmitted. Through these steps, the explanation unit can provide medical professionals with the logic and evidence behind the model's predictions, enhance the transparency and trust of the model, and thus play a greater role in clinical decision-making.

[0025] As an optional embodiment: the weight corresponding to each layer of the large language model is obtained and attention score The specific steps are as follows: For each layer, the corresponding weight , determine the layer index i to be queried in the model parameter list to locate the weight parameters of the layer; extract and record the weight parameter values ​​of the layer, which are learned during the model training process; During the forward propagation of the model, the query, key, and value are first calculated. The dot product of the query and the key is calculated to obtain the original attention score. The original attention score is scaled, usually divided by the square root of the key vector dimension, and the softmax function is applied to normalize the scaled attention score to obtain the final attention weight.

[0026] As an optional embodiment: the specific working steps of the interpretation unit also include: After the fine-tuned large language model generates a prediction result, all features Total importance score Arrange in descending order and take out the total importance score The maximum and minimum values ​​of are the features that have the greatest and least impact on model prediction. The sample data after removing the features is input into the fine-tuned model again to obtain new output results. The new output result is also used as the decision process of the fine-tuned large language model and transmitted.

[0027] It should be noted that by identifying and removing the features that have the greatest impact on model predictions, the internal working mechanism of the model can be revealed and the transparency of the model can be enhanced.

[0028] As an optional embodiment: the specific working steps of the correction module are as follows: The model output results and output changes after removing features are generated based on the output changes to show which feature changes will affect the model's prediction results. The counterfactual explanation is also output as the decision process of the fine-tuned large language model for reference by staff.

[0029] It should be noted that counterfactual explanations provide a way to explain the model's decision-making process by showing how small changes in input features affect model predictions. In the medical field, the model's decision-making process needs to comply with ethical and legal standards, and counterfactual explanations can help ensure that the model's decision-making process is fair and unbiased.

[0030] How it works It should be noted that, first, the model loading module is responsible for importing the large-scale pre-trained language model open source on the market and executing the initialization code to ensure that the model can be loaded from the pre-trained state and is ready for fine-tuning. The preprocessing module receives the original medical text data, including medical records and medical reports, and cleans, annotates and formats them to generate training and verification data sets suitable for model input. Then, the fine-tuning module contains two sub-units: the fine-tuning unit receives the pre-processed data, obtains the adjustment parameters of the model through training, and then fine-tunes the model to generate a fine-tuned large language model; the interpretation unit is responsible for outputting the decision process of the fine-tuning model and providing transparency. Finally, the correction module receives the decision process provided by the interpretation unit and generates evaluation indicators based on it, and further outputs the corrected large language model based on these indicators to ensure the accuracy and fairness of the model. Through the fine-tuning module, the fine-tuning unit receives the preprocessed data and fine-tunes the model, while the interpretation unit outputs the decision-making process of the model so that medical professionals can understand the internal logic of the model. It not only improves the adaptability and accuracy of the model, but also enhances the trust of medical professionals in the AI ​​system, ensuring the rationality and safety of clinical decisions. At the same time, the correction module receives the decision-making process of the interpretation unit, generates evaluation indicators based on it, and outputs the corrected large language model, which effectively reduces model bias and improves the fairness and ethics of medical decisions.

[0031] The above are only preferred implementations of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical staff in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of this template.

Claims

1. A large language model fine-tuning system, characterized in that: A model loading module is included, wherein the model loading module is used to load a pre-adjusted large language model; A preprocessing module, which is used to process the original medical text data and transmit the processed data; A fine-tuning module, the fine-tuning module comprising a fine-tuning unit and an interpretation unit, the fine-tuning unit being used to receive the processed data transmitted by the preprocessing module, train to obtain adjustment parameters of the large language model of the model loading module, and adjust the large language model of the model loading module according to the adjustment parameters to obtain a fine-tuned large language model; The interpretation unit is used to output the decision process of the fine-tuned large language model and transmit it; A correction module is used to correct the output result of the fine-tuned large language model and transmit the correction result as an auxiliary decision.

2. A large language model fine-tuning system according to claim 1, characterized in that: The specific working steps of the model loading module are as follows: Identify the open source large language models to use; Prepare the Python environment and install the necessary package manager using pip; Download the required model files from the open source platform, including large language model weights and configuration files; Use a deep learning framework to load large language model weights; Set large language model parameters, including learning rate, optimizer selection, and number of fine-tuning layers; Get the pre-tuned large language model.

3. A large language model fine-tuning system according to claim 2, characterized in that: The specific working steps of the preprocessing module are as follows: Obtaining original medical text data and classifying the original medical text data, specifically including electronic medical records, medical imaging reports, medical question and answer data, and clinical data; Then, according to the classification results of the original medical text data, each type of original medical text data is preprocessed, and the processed data is transmitted to the corresponding micro-carving module.

4. A large language model fine-tuning system according to claim 3, characterized in that: The specific steps of preprocessing each type of original medical text data according to the classification results of the original medical text data are as follows: For the original medical text data of electronic medical records, we first remove the data of headers, footers, and template tags, and then standardize the date, time, and measurement units. We use named entity recognition technology to identify patient information and diagnosis results in the medical records, and use the mean or median to fill in missing values ​​to complete the preprocessing. For the original medical text data of medical imaging reports, first, descriptive text is extracted from the imaging report, the diagnostic conclusion is extracted as key information, and the extracted text is vectorized and converted into vector form to complete the preprocessing; For the original medical text data of the medical question and answer data type, first separate the questions and answers from the question and answer pairs, and standardize the questions and answers to complete the preprocessing; For raw medical text data of the clinical data type, clinical trial data from different sources are integrated into one dataset, and the data are converted into a unified format to complete preprocessing.

5. A large language model fine-tuning system according to claim 4, characterized in that: The specific working steps of the fine-tuning unit are as follows: Receiving the processed data transmitted by the preprocessing module, and dividing the processed data into a training set X and a verification set Y; Load a large pre-trained language model and set key hyperparameters during fine-tuning, including learning rate, batch size, and number of training epochs; Use the preprocessed data to train the model and update the weights of the large language model. The specific calculation steps are: During the training process, the model is evaluated regularly using the validation set, and early stopping parameters are set, including the minimum change to be monitored, the maximum number of iterations allowed, and performance metrics. When the performance on the validation set no longer improves, training is stopped to obtain a fine-tuned large language model.

6. A large language model fine-tuning system according to claim 5, characterized in that: The specific working steps of the interpretation unit include: Receiving the fine-tuned large language model and pre-input feature samples, inputting the pre-input feature samples into the large language model, and generating a prediction result of the large language model; The number of layers n in the fine-tuned large language model in the process of generating prediction results; Each feature in the pre-input feature sample in the large language model is labeled as , Indicates Features Get the weight corresponding to each layer of the large language model and attention score , weight and attention score Expressed as The weights and Attention scores; Multiply the weight of each layer by the attention score to get the feature Contributions of importance; Add up the contributions of all layers to get the feature Total importance score , Indicates The total importance score of the features; All features Total importance score As the decision process of the fine-tuned large language model, and transfer.

7. A large language model fine-tuning system according to claim 6, characterized in that: The weight corresponding to each layer of the large language model is obtained and attention score The specific steps are as follows: For each layer, the corresponding weight , determine the layer index i to be queried in the model parameter list to locate the weight parameters of the layer; extract and record the weight parameter values ​​of the layer, which are learned during the model training process; During the forward propagation of the model, the query, key, and value are first calculated. The dot product of the query and the key is calculated to obtain the original attention score. The original attention score is scaled, usually divided by the square root of the key vector dimension, and the softmax function is applied to normalize the scaled attention score to obtain the final attention weight.

8. The large language model fine-tuning system according to claim 6, characterized in that: The specific working steps of the interpretation unit also include: After the fine-tuned large language model generates a prediction result, all features Total importance score Arrange in descending order and take out the total importance score The maximum and minimum values ​​of are the features that have the greatest and least impact on model prediction. The sample data after removing the features is input into the fine-tuned model again to obtain new output results. The new output result is also used as the decision process of the fine-tuned large language model and transmitted.

9. The large language model fine-tuning system according to claim 1, characterized in that: The specific working steps of the correction module are as follows: The model output results and output changes after removing features are generated based on the output changes to show which feature changes will affect the model's prediction results. The counterfactual explanation is also output as the decision process of the fine-tuned large language model for reference by staff.

Images