LLM Fine-Tuning With Masked Options for Multiclass Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models perform inadequately in multiclass classification tasks, lacking the ability to effectively learn why incorrect options are incorrect, leading to suboptimal performance and accuracy.
Innovation Solution
A method and apparatus for fine-tuning large language models by masking candidate option contents in training data, calculating losses based on predicted and correct option numbers and contents, and optimizing the model to minimize total loss, thereby enhancing the learning of correct and incorrect options.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large language models are applied to multiclass classification tasks using conventional training methods, then the models can perform basic classification, but the accuracy and performance are suboptimal because the models cannot effectively learn why incorrect options are incorrect
Solution Approach 1:
The training data is segmented into two distinct parts: first training data containing question, candidate options, and correct answer; second training data obtained by masking the correct option content. This segmentation allows the model to learn different aspects through different data types, improving both accuracy and reliability in multiclass classification tasks
Solution Approach 2:
A masking mechanism is introduced as an intermediary in the training process. The masking operation hides the correct option content in the second training data, forcing the model to rely on learning from both masked and unmasked data, thereby improving its ability to distinguish correct from incorrect options
2Adaptability or versatility
If conventional fine-tuning methods are used for multiclass classification, then the model can be adapted to specific tasks, but the complexity of the predicted result and time consumption in downstream tasks remain high
Solution Approach 1:
The training approach changes the parameters being optimized by introducing dual-loss training: one loss for predicting the correct option number and another for predicting the masked option content. This parameter change in training objectives leads to simpler and more efficient predictions in downstream tasks
3Adaptability or versatility
If conventional fine-tuning methods are used for multiclass classification, then the model can be adapted to specific tasks, but the time consumption for obtaining predicted results in downstream tasks is excessive
Solution Approach 1:
The model performs preliminary learning during training by simultaneously learning to predict both the option number and the option content through masked language modeling. This preliminary action of learning multiple aspects during training reduces the time required for obtaining accurate predictions in downstream tasks
Data Source
AI summary
A method of fine-tuning a large language model includes steps of obtaining training data containing first training data and second training data, the first training data being inclusive of a question, a plurality of candidate option numbers and their corresponding candidate option contents, and a correct option number and its corresponding correct option content, the correct option number and its corresponding correct option content being one of the plurality of the candidate option numbers and their corresponding candidate option contents, respectively, and the second training data being obtained by masking a candidate option content in the first training data; inputting the training data into the large language model to generate a predicted result by utilizing the large language model; and optimizing the large language model based on the predicted result as well as the correct option number and its corresponding correct option content.