Large model fine tuning training system

Abstract

The invention discloses a large model fine-tuning training system, and belongs to the technical field of education and teaching model training, in the system, a data acquisition subsystem is responsible for collecting a data set containing training and verification question-answer pairs related to the humanity and community field, and the data is from professional textbooks, academic papers and practice cases; the model fine tuning subsystem adopts a small sample learning method to carry out training and fine tuning on the selected general large model, and outputs the fine-tuned model; and the fine-tuned model verification subsystem evaluates the performance of the fine-tuned model, and determines whether the model needs to be further optimized according to an evaluation result so as to ensure the accuracy and practicability of the model. According to the method, the precision and practicability of the model in the field of education and teaching are improved, and the quality and efficiency of the final model are improved by performing screening and difficulty grading on the data, so that the model better adapts to specific education and teaching application scenes and requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of educational teaching model training, and specifically relates to a large model fine-tuning training system. Background Art

[0002] When general large language models are used in actual vertical business scenarios, their general capabilities sometimes become an application barrier. This is because general models are usually trained on a wide range of data sets rather than data for specific fields, which may lead to their insufficient performance when dealing with complex problems in specific fields; and general models lack a sufficiently deep understanding of certain industry-specific knowledge, terminology, and processes. For example, in the medical or financial fields, precise understanding of professional terminology and concepts is crucial to providing accurate answers, and general models may lack a deep understanding of these professional fields.

[0003] In addition, general-purpose models may require a lot of hardware resources to deploy and run, which may be an obstacle for some application scenarios. Moreover, general-purpose models may not be able to obtain the latest domain information because their training data may not be updated in real time. These limitations indicate that although general-purpose large language models have achieved remarkable results in a variety of natural language processing tasks, they may need to be combined with domain-specific knowledge graphs in professional question-answering systems to improve the accuracy and relevance of question answering.

[0004] The general model can be applied to large models related to education and teaching, such as the "AI Talent" education and teaching-specific large model. This large model aims to explore and try to solve the difficulties in the construction of new liberal arts in the past, such as the high technical threshold of the integration of arts and sciences, the disconnection between experiments and practicality, and the difficulty of integrating into the local ecology through the deep integration of artificial intelligence technology and philosophy and social science higher education teaching. It will form the precipitation, accumulation, sharing, innovation and inheritance of knowledge data among eight disciplines, including economics, management, literature, law, science, engineering, education, and art, to help cultivate high-quality top innovative talents in new finance and economics.

[0005] Therefore, a new system for fine-tuning large models is needed to better adapt to specific professional requirements and actual situations such as education and teaching, and to solve the problem of insufficient performance of traditional general models in fields such as education and teaching. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a large model fine-tuning training system, which improves the accuracy and practicality of the model in the field of education and teaching, thereby effectively overcoming the performance deficiencies that traditional general models may encounter in the field of education and teaching.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a large model fine-tuning training system, including a data acquisition subsystem, a model fine-tuning subsystem and a fine-tuned model verification subsystem, wherein: The data acquisition subsystem is used to acquire training data and verification data in the field of humanities and social sciences. The training data includes a number of training question-training answer pairs, and the verification data includes a number of verification question-verification answer pairs. The model fine-tuning subsystem is used to determine the general large model to be fine-tuned, and train and fine-tune the determined general large model based on training data and small sample learning, and output the fine-tuned large model for the humanities and social sciences; The fine-tuned model verification subsystem is used to evaluate the performance of the fine-tuned large model based on the verification data, and determine whether the fine-tuned large model needs to be optimized based on the evaluation results: If optimization is not required, the fine-tuned large model after verification is output; If optimization is required, the initial parameters of the determined general large model are reset and transmitted to the model fine-tuning subsystem, which then re-fine-tunes the model based on the training data obtained in the humanities and social sciences.

[0008] Preferably, the obtaining of training data comprises the following steps: Obtain data to be screened related to the content in the humanities and social sciences, including questions to be screened and answers to the questions to be screened. The content in the humanities and social sciences includes professional textbooks, academic papers and internship training cases related to economics, management, literature, law, science, engineering, education and art; Randomly grouping the data to be screened to obtain several data groups to be screened; Performing intra-group screening on each data group to be screened to obtain a screened data group; Determine the most similar post-screening data group pair, perform secondary screening on the post-screening data group pair, and obtain a secondary post-screening data group; The data group after secondary screening and the data group after screening without secondary screening are split to obtain training data.

[0009] Preferably, the step of obtaining the screened data set comprises the following steps: Randomly select any one of the questions to be screened in the data group to be screened, obtain the text keyword combination to be screened of the question to be screened and save it to the text keyword set corresponding to the data group to be screened; Obtain the text keywords to be screened of the next question to be screened in the data group to be screened one by one and record them as the text keyword combination to be compared; Perform similarity comparison between the text keyword combination to be compared and the text keyword combination to be screened in the text keyword set one by one: If the text keyword combination to be compared is similar to any text keyword combination to be screened in the text keyword set, the text keyword combination to be compared is deleted and the question to be screened corresponding to the text keyword combination to be compared and the answer to the question to be screened are deleted; If the text keyword combination to be compared is not similar to any of the text keyword combinations to be screened in the text keyword set, the text keyword combination to be compared is saved in the text keyword set, and the question to be screened corresponding to the text keyword combination to be compared and the answer to the question to be screened are retained; The data to be screened after the in-group screening is recorded as a screened data group, and the text keyword set corresponding to the screened data group is saved.

[0010] Preferably, the determining of the most similar screened data set pair, performing secondary screening on the screened data set pair, and obtaining the secondary screened data set comprises the following steps: Perform similarity judgment on the text keyword sets corresponding to any two filtered data groups: If two text keyword sets are not similar, the corresponding two filtered data sets will not be screened again; If two text keyword sets are similar, then the corresponding two filtered data groups are the most similar filtered data group pairs; determining the number of to-be-screened questions in two screened data sets in the most similar pair of screened data sets; The large number of filtered data sets is used as the benchmark, and the questions to be filtered in the small number of filtered data sets are compared with the questions to be filtered in the large number of filtered data sets for similarity: If they are similar, delete the unscreened questions in the screened data group with a small number; If they are not similar, the unscreened problems in the screened data group with a small number are retained; Output the two filtered data sets to obtain a secondary filtered data set.

[0011] Preferably, the step of determining the general large model to be fine-tuned comprises the following steps: The training data is grouped according to difficulty levels to obtain first difficulty training data, second difficulty training data, and third difficulty training data; Input the training questions in the first difficulty training data, the second difficulty training data and the third difficulty training data into each candidate universal large model one by one, and obtain the answer evaluation data of each candidate universal large model, the answer evaluation data including the first difficulty answer evaluation data, the second difficulty answer evaluation data and the third difficulty answer evaluation data; Determine the general large model to be fine-tuned based on the response evaluation data.

[0012] Preferably, the difficulty level grouping of the training data comprises the following steps: Obtain training question keyword combinations corresponding to each training question in the training data; Obtain the keyword combination sets of each difficulty level stored in the database, and determine whether the training question keyword combination hits one of the keyword combination sets of each difficulty level: If there is a hit, then the difficulty coefficient corresponding to the hit difficulty level keyword combination set stored in the database is obtained; If there is no hit, the difficulty coefficient corresponding to the keyword combination of the training question is 0; Get the correct rate of human responses to each training question; Determine the difficulty distribution coefficient for each training question based on the difficulty coefficient and the correct answer rate: ; In the formula, A is the difficulty distribution coefficient, Nx is the difficulty coefficient, DL is the correct rate of human response, and a is The weight factor of Nx stored in the database is The weight factor of DL stored in the database; If the difficulty allocation coefficient is less than the first difficulty allocation threshold stored in the database, the training question-training answer pair corresponding to the difficulty allocation coefficient is the first difficulty training data; If the difficulty allocation coefficient is not less than the first difficulty allocation threshold stored in the database, but less than the second difficulty allocation threshold stored in the database, then the training question-training answer pair corresponding to the difficulty allocation coefficient is the second difficulty training data; If the difficulty allocation coefficient is not less than the second difficulty allocation threshold stored in the database, the training question-training answer pair corresponding to the difficulty allocation coefficient is the third difficulty training data.

[0013] Preferably, determining the general large model to be fine-tuned based on the response evaluation data comprises the following steps: Obtaining first difficulty response parameter data, second difficulty response parameter data, and third difficulty response evaluation data stored in a database; Based on the first difficulty answer evaluation data and the first difficulty answer parameter data, the first difficulty answer performance index corresponding to the candidate general large model is obtained. The first difficulty answer parameter data includes the first difficulty answer accuracy rate, the first difficulty correct answer completeness rate, the first difficulty correct answer reaction time and the first difficulty correct answer non-empty rate. The first difficulty answer parameter data includes the first difficulty parameter answer accuracy rate, the first difficulty parameter correct answer completeness rate, the first difficulty parameter correct answer reaction time and the first difficulty parameter correct answer non-empty rate; Based on the second difficulty answer evaluation data and the second difficulty answer parameter data, the second difficulty answer performance index corresponding to the candidate general large model is obtained, and the second difficulty answer parameter data includes the second difficulty answer accuracy rate, the second difficulty correct answer completeness rate, the second difficulty correct answer reaction time, and the second difficulty correct answer non-empty rate. The second difficulty answer parameter data includes the second difficulty parameter answer accuracy rate, the second difficulty parameter correct answer completeness rate, the second difficulty parameter correct answer reaction time, and the second difficulty parameter correct answer non-empty rate; Based on the third difficulty response evaluation data and the third difficulty response parameter data, the third difficulty response performance index corresponding to the candidate general large model is obtained. The third difficulty response parameter data includes the third difficulty response accuracy rate, the third difficulty correct answer completeness rate, the third difficulty correct answer reaction time and the third difficulty correct answer non-empty rate. The third difficulty response parameter data includes the third difficulty parameterized answer accuracy rate, the third difficulty parameterized correct answer completeness rate, the third difficulty parameterized correct answer reaction time and the third difficulty parameterized correct answer non-empty rate; The first difficulty response performance index, the second difficulty response performance index and the third difficulty response performance index are weighted and summed to obtain the overall performance effect value of the large model; The candidate general large model corresponding to the smallest overall performance effect value of the large model is determined as the general large model to be fine-tuned.

[0014] Preferably, the method of training and fine-tuning the determined general large model based on the training data and small sample learning, and outputting the fine-tuned large model, comprises the following steps: Initialize the fine-tuning parameters of the determined general large model, including the learning rate, the batch size of the input training data, and the number of iterations; Create task prompts and construct model inputs; Load the determined general large model; Calculate the loss based on the cross entropy loss function; Based on the results of the loss calculation, the gradient descent algorithm adjusts the weights of the model through the Adam optimizer to reduce the loss value; After completing the number of iterations, the fine-tuned large model is output.

[0015] Preferably, the performance evaluation of the fine-tuned large model based on the verification data comprises the following steps: Input the verification questions in the verification data into the fine-tuned large model one by one to obtain the output answer set; Based on the verification question-verification answer pair, the answer set is analyzed to obtain verification result data, which includes the verification result correctness rate, the correct verification result completeness rate and the correct verification result non-void rate; Obtaining hardware system operation data during the verification process, the hardware system operation data including the number of hardware system crashes during the verification process, the energy consumption during the verification process, and the memory usage during the verification process; Acquire verification result allowable deviation data, the verification result allowable deviation data including a correct verification result completeness rate allowable deviation value and a correct verification result non-void rate allowable deviation value; Acquire the verification requirement performance data stored in the database, the verification requirement performance data including the required accuracy rate of the verification result, the required completeness rate of the correct verification result, the required non-void rate of the correct verification result, the required energy consumption during the verification process, and the required memory usage during the verification process; Determine whether the number of hardware system crashes during the verification process is 0: If it is not 0, the evaluation result of the performance of the large model after fine-tuning is that it needs to be optimized; If it is not 0, the verification index is obtained based on the verification result data, the hardware system operation data, the verification result allowable deviation data and the verification requirement performance data; If the validation index is greater than the validation threshold stored in the database, the evaluation result of the performance of the large model after fine-tuning is that it needs to be optimized; If the validation index is not greater than the validation threshold stored in the database, the evaluation result of the performance of the large model after fine-tuning is that no optimization is required.

[0016] Preferably, obtaining the verification result allowable deviation data comprises the following steps: Obtaining a first difficulty response performance index, a second difficulty response performance index, and a third difficulty response performance index and recording them as comparison data; Acquire each matching data stored in the database, the matching data including a first difficulty answer performance index matching value, a second difficulty answer performance index matching value, and a third difficulty answer performance index matching value; Compare the comparison data with the matching data one by one to determine the matching data that is most similar to the comparison data; The verification result allowable deviation data corresponding to the most similar matching data stored in the database is obtained.

[0017] The present invention has the following beneficial effects: By using high-quality, specialized training and verification data extracted from professional textbooks, academic papers, and actual cases, the present invention can refine and optimize the learning ability of the model, making it better adapted to the professional requirements and actual conditions of education and teaching. This method improves the accuracy and practicality of the model in the field of education and teaching, thereby effectively overcoming the performance deficiencies that traditional general models may encounter in education and teaching applications.

[0018] The present invention ensures that each step of data processing is to improve the quality and efficiency of the final model by screening the data and grading the difficulty, so that the model can better adapt to the application scenarios and needs of education and teaching. This strategy is particularly suitable for complex model scenarios that require extracting the most useful information related to education and teaching from a large amount of data for efficient training. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the subsystem structure of the system of the present invention; Figure 2 Flowchart for determining a general macro model to be fine-tuned for the present invention. DETAILED DESCRIPTION

[0020] The embodiments of the present invention are further described below in conjunction with the accompanying drawings: like Figure 1 As shown, the large model fine-tuning training system includes a data acquisition subsystem, a model fine-tuning subsystem and a post-fine-tuning model verification subsystem, wherein: the data acquisition subsystem is used to obtain training data and verification data in the humanities and social sciences, the training data includes several training question-training answer pairs, and the verification data includes several verification question-verification answer pairs.

[0021] Obtain the data to be screened related to the humanities and social sciences. These data contain question and answer pairs in professional fields (economics, management, literature, law, science, engineering, education, art, etc.), providing the model with application cases and professional knowledge in actual situations. The data to be screened include questions to be screened and the answers to the questions to be screened. It should be noted that the data to be screened can be subdivided into economics-related data to be screened, management-related data to be screened, and literature-related data to be screened, etc. These data to be screened can be obtained from professional textbooks, academic papers, and internship training cases in various humanities and social sciences fields; the data to be screened are randomly grouped to obtain several data groups to be screened. It should be noted that random grouping can group economics-related data to be screened or other subject-related data to be screened into similar subjects, or can group all subject-related data to be screened together so that the grouped data to be screened can contain content from multiple subjects; each data group to be screened is screened within the group to obtain a screened data group; each data group is screened independently based on standards such as data quality and relevance, which helps to extract the most representative and practical data from each group.

[0022] Determine the most similar pairs of screened data groups, perform secondary screening on the screened data group pairs, and obtain secondary screened data groups; compare different screened data groups, and find the data group with the highest similarity for further fine screening. This step can further refine the data and ensure that the obtained training data group has a high degree of professional correspondence and learning value. Split the secondary screened data group and the non-secondary screened data group to obtain training data. The secondary screened data group and the non-secondary screened data group are used for model training respectively. Such a division can maximize the use of data in the model training process while maintaining the wide coverage and depth of the data set.

[0023] Through a multi-stage screening process, the system is able to select the most relevant and informative data from a large amount of candidate data related to the humanities and social sciences. This ensures the quality of the training data, thereby improving the efficiency of the model in question-answering training in the humanities and social sciences and the performance of the final model. Through random grouping and in-group screening, the diversity of the data is guaranteed, which helps the model learn a wider range of problem-solving strategies and enhances its generalization ability in applications in the humanities and social sciences education field.

[0024] A careful screening process can reduce noise and irrelevant information in the data, which helps prevent the model from overfitting to the training data (i.e., overfitting) during training and failing to perform well on new, unseen data. By precisely controlling the selection and use of data, computing resources can be used more efficiently, which is especially important when the amount of data is large and the computing cost is high.

[0025] Randomly select any one of the questions to be screened in the data group to be screened, obtain the text keyword combination to be screened of the question to be screened and save it in the text keyword set corresponding to the data group to be screened; randomly select a question from the data group to be screened as the starting point, and extract keywords. Such randomness helps to avoid initial bias and ensure the representativeness of the keyword set.

[0026] The next question to be screened in the data set to be screened is obtained one by one and recorded as the text keyword combination to be compared; by comparing the keywords of each new question with the existing keyword set, the system can effectively identify and eliminate duplicate or highly similar questions. This step ensures that each question is unique and increases the information entropy of the training data.

[0027] The similarity of the text keyword combination to be compared is compared with the text keyword combination to be screened in the text keyword set one by one: if the text keyword combination to be compared is similar to any text keyword combination to be screened in the text keyword set, the text keyword combination to be compared is deleted and the corresponding question to be screened and the answer to the question to be screened are deleted; if the text keyword combination to be compared is not similar to any text keyword combination to be screened in the text keyword set, the text keyword combination to be compared is saved in the text keyword set, and the corresponding question to be screened and the answer to the question to be screened are retained; whenever a new and unique keyword combination is found, it is added to the set. This continuous updating process helps to keep the dataset fresh and relevant.

[0028] The data to be screened after the group screening is recorded as the screened data group, and the text keyword set corresponding to the screened data group is saved. If the keyword combination of a question is considered to be repeated, the related questions and answers will be deleted to avoid wasting resources on repeated or similar questions. If the keyword combination is unique, the question and answer will be retained to provide more angles of data for model training.

[0029] By removing questions and answers with similar keyword combinations, the system reduces data redundancy and ensures that each retained data point brings independent value to model training. This not only improves the quality of training data, but also improves the model's ability to handle new questions. Retaining questions with dissimilar keyword combinations enhances the diversity of the data set. Such diversity is crucial to building a robust model and can help the model perform more stably when encountering different types of questions.

[0030] The similarity of the text keyword sets corresponding to any two filtered data groups is judged: if the two text keyword sets are not similar, the corresponding two filtered data groups are not screened for the second time; if the two text keyword sets are similar, the corresponding two filtered data groups are the most similar filtered data group pair; the number of questions to be screened in the two filtered data groups in the most similar filtered data group pair is determined; the filtered data group with a larger number is used as a benchmark, and the questions to be screened in the other filtered data group with a smaller number are compared for similarity with the questions to be screened in the filtered data group with a larger number (through the combination of text keywords to be screened corresponding to the questions to be screened): if they are similar, the questions to be screened in the filtered data group with a smaller number are deleted; if they are not similar, the questions to be screened in the filtered data group with a smaller number are retained; the two filtered data groups are output to obtain a secondary filtered data group.

[0031] By performing a secondary screening of the most similar data groups, the data set can be further streamlined to remove duplicate or overly similar data, ensuring that the model training set contains the most representative and useful information. By reducing the training of duplicate data, computing resources and time can be saved, making the training process more efficient.

[0032] The model fine-tuning subsystem is used to determine the general large model to be fine-tuned, and train and fine-tune the determined general large model based on training data and small sample learning, and output the fine-tuned large model. The small sample learning method is used to conduct targeted training on the pre-trained general large model. This step adjusts the parameters of the model through specialized training data, so that the model can better understand and handle specific problems in the professional field.

[0033] like Figure 2 As shown, the training data are grouped according to difficulty levels to obtain first difficulty training data, second difficulty training data and third difficulty training data; the training questions in the first difficulty training data, the second difficulty training data and the third difficulty training data are input into each candidate universal big model one by one to obtain the reply evaluation data of each candidate universal big model, the reply evaluation data includes first difficulty reply evaluation data, second difficulty reply evaluation data and third difficulty reply evaluation data; the universal big model to be fine-tuned is determined based on the reply evaluation data.

[0034] By grading the difficulty of the training data, we can see how different candidate general large models perform in the humanities and social sciences before fine-tuning, so that we can screen out suitable general large models to be fine-tuned. In addition, by grading the difficulty of the training data, more targeted model training can be achieved. The model can learn gradually based on data of different difficulty levels, from simple to complex, and gradually adapt to more difficult questions. This progressive training helps improve the learning efficiency and final performance of the model.

[0035] The difficulty level grouping of the training data includes the following steps: obtaining the training question keyword combination corresponding to each training question in the training data; obtaining the keyword combination set of each difficulty level stored in the database (the relevant professionals, such as on-the-job professors, etc., rate the difficulty of different keyword combinations, and construct the keyword combinations of the same difficulty level into keyword combination sets of each difficulty level), and determining whether the training question keyword combination hits one of the difficulty level keyword combination sets (keyword similarity matching): if it hits, then the difficulty coefficient corresponding to the hit difficulty level keyword combination set stored in the database is obtained; if it does not hit, then the difficulty coefficient corresponding to the training question keyword combination is 0; and obtaining the correct rate of human responses to each training question (the correct response rate of each question can be collected through a large number of written or online tests). Combined with the actual response accuracy, the difficulty level of the question can be further adjusted or used to evaluate the performance of the model in practical applications.

[0036] Determine the difficulty distribution coefficient for each training question based on the difficulty coefficient and the correct answer rate: ; In the formula, A is the difficulty distribution coefficient, Nx is the difficulty coefficient, DL is the correct rate of human response, and a is The weight factor of Nx stored in the database is The weight factor of DL stored in the database.

[0037] If the difficulty distribution coefficient is less than the first difficulty distribution threshold stored in the database, the training question-training answer pair corresponding to the difficulty distribution coefficient is the first difficulty training data; if the difficulty distribution coefficient is not less than the first difficulty distribution threshold stored in the database, but less than the second difficulty distribution threshold stored in the database, the training question-training answer pair corresponding to the difficulty distribution coefficient is the second difficulty training data; if the difficulty distribution coefficient is not less than the second difficulty distribution threshold stored in the database, the training question-training answer pair corresponding to the difficulty distribution coefficient is the third difficulty training data.

[0038] Before any form of fine-tuning is performed, it is very important to understand how the model reacts to and solves problems of varying difficulty in its untuned state. This provides a clear baseline to help evaluate the effectiveness and necessity of fine-tuning. By comparing the performance of different candidate models on training data of various difficulty levels, those models that perform best on a specific set of problems can be selected for further fine-tuning. This not only improves the efficiency of the fine-tuned model, but also ensures that resources are invested in the candidates that are most likely to succeed.

[0039] The first difficulty response parameter data, the second difficulty response parameter data and the third difficulty response evaluation data stored in the database are obtained. Finally, based on the response evaluation data, a suitable general large model to be fine-tuned is selected to ensure that the selected model can perform well when dealing with practical problems.

[0040] The first difficulty answer performance index corresponding to the candidate general large model is obtained based on the first difficulty answer evaluation data and the first difficulty answer parameter data. The first difficulty answer parameter data include the first difficulty answer accuracy rate (the ratio of the number of correctly answered questions to the total number of questions), the first difficulty correct answer completeness rate (the ratio of the number of keywords in the correctly answered answers to the required number of keywords), the first difficulty correct answer reaction time (the time from the completion of question input to the start of answer output) and the first difficulty correct answer non-empty rate (the ratio of the number of correct keyword words to the total number of output words). The first difficulty answer parameter data include the first difficulty parameter answer accuracy rate, the first difficulty parameter correct answer completeness rate, the first difficulty parameter correct answer reaction time and the first difficulty parameter correct answer non-empty rate.

[0041] ; Where i is the number of the candidate general large model, Fzs i is the correct answer rate of the first difficulty of the i-th candidate general large model, Fws i is the completeness rate of the correct answers of the first difficulty of the i-th candidate general large model, Fts i is the reaction time of the first difficulty correct answer of the i-th candidate general large model, Fks i is the non-empty rate of the first difficulty correct answer of the i-th candidate general large model, Fzc is the correct rate of the first difficulty parameter answer, Fwc is the complete rate of the first difficulty parameter correct answer, Ftc is the reaction time of the first difficulty parameter correct answer, Fkc is the non-empty rate of the first difficulty parameter correct answer, Fr i is the first difficulty answer performance index of the i-th candidate general large model.

[0042] Based on the second difficulty answer evaluation data and the second difficulty answer parameter data, the second difficulty answer performance index corresponding to the candidate general large model is obtained. The second difficulty answer parameter data include the second difficulty answer accuracy rate, the second difficulty correct answer completeness rate, the second difficulty correct answer reaction time and the second difficulty correct answer non-empty rate. The second difficulty answer parameter data include the second difficulty parameter answer accuracy rate, the second difficulty parameter correct answer completeness rate, the second difficulty parameter correct answer reaction time and the second difficulty parameter correct answer non-empty rate.

[0043] ; In the formula, Szs iis the correct answer rate of the second difficulty of the i-th candidate general large model, Sws i is the completeness rate of the correct answers of the second difficulty of the i-th candidate general large model, Sts i Sks is the reaction time of the second-difficulty correct answer of the i-th candidate general large model, i is the non-empty rate of the second difficulty correct answer of the i-th candidate general large model, Szc is the correct rate of the second difficulty parameter answer, Swc is the complete rate of the second difficulty parameter correct answer, Stc is the reaction time of the second difficulty parameter correct answer, Skc is the non-empty rate of the second difficulty parameter correct answer, Sr i is the second difficulty response performance index of the i-th candidate general large model.

[0044] Based on the third difficulty answer evaluation data and the third difficulty answer parameter data, the third difficulty answer performance index corresponding to the candidate general large model is obtained. The third difficulty answer parameter data include the third difficulty answer accuracy rate, the third difficulty correct answer completeness rate, the third difficulty correct answer reaction time and the third difficulty correct answer non-empty rate. The third difficulty answer parameter data include the third difficulty parameter answer accuracy rate, the third difficulty parameter correct answer completeness rate, the third difficulty parameter correct answer reaction time and the third difficulty parameter correct answer non-empty rate.

[0045] ; In the formula, Tzs i is the correct answer rate of the third difficulty of the i-th candidate general large model, Tws i is the completeness rate of correct answers of the third difficulty level of the i-th candidate general large model, Tts i is the reaction time of the correct answer of the third difficulty of the i-th candidate general large model, Tks i is the non-empty rate of the third difficulty correct answer of the i-th candidate general large model, Tzc is the correct rate of the third difficulty parameter answer, Twc is the complete rate of the third difficulty parameter correct answer, Ttc is the reaction time of the third difficulty parameter correct answer, Tkc is the non-empty rate of the third difficulty parameter correct answer, Tr i is the second difficulty response performance index of the i-th candidate general large model.

[0046] By comprehensively considering the accuracy, answer completeness, response speed, and answer quality (non-empty rate), the formula can comprehensively evaluate the comprehensive ability of the model in dealing with practical problems, rather than just a single accuracy rate. The parameter data in the formula is used as a benchmark value, so that the model is evaluated relative to a set of preset standards, which helps to compare the performance of different models fairly. By calculating the response performance index of each model, the model with the best performance at the first difficulty level can be intuitively compared and selected for further fine-tuning and application, thereby improving the quality of the final product and user satisfaction.

[0047] The natural logarithm is used in the formula to smooth the impact of the indicator and avoid excessive influence of a certain extreme value on the total score. Logarithmic transformation can help reduce the distortion caused by extreme values ​​and make the score more stable and reasonable. Each comparison item in the formula actually standardizes the performance of the model with the preset benchmark (parameter data). This comparison helps to maintain consistency and comparability of evaluation in different tests.

[0048] A high accuracy rate is a basic indicator of good model performance, but it needs to be combined with other parameters to evaluate the completeness of the answer and the quality of the response. The completeness rate supplements the limitations of the accuracy rate, ensuring that the model not only gives the correct answer, but also the answer information is comprehensive. Fast responses must balance accuracy and completeness, ensuring that the quality of the answer is not sacrificed while answering quickly. The non-emptiness rate emphasizes improving the density and quality of information while maintaining the accuracy and completeness of the answer, avoiding lengthy and invalid answers.

[0049] Together, these parameters form a multi-dimensional model evaluation framework that considers not only the basic accuracy of the model in answering questions, but also the quality, efficiency, and information density of the answers. This comprehensive evaluation provides a more comprehensive understanding of the model's performance in actual operations, ensuring that the selected model not only responds quickly, but also provides answers that are both accurate and informative, and have practical value. This evaluation method helps to discover and optimize potential deficiencies in the model, so that it can better adapt to specific application needs.

[0050] The first difficulty response performance index, the second difficulty response performance index and the third difficulty response performance index are weightedly summed to obtain the overall performance effect value of the large model; the candidate general large model corresponding to the smallest large model overall performance effect value is determined as the general large model to be fine-tuned.

[0051] ; In the formula, Zx i is the overall performance value of the large model of the i-th candidate general large model, c 1 Fr stored in the database i The weight factor, c 2 Sr stored in the databasei The weight factor, c 3 Tr stored in the database i The weight factor, c 1 , c 2 and c 3 The sum of is 1.

[0052] Initialize the fine-tuning parameters of the determined general large model, including the learning rate, input training data batch size and number of iterations; create task prompts and build model inputs; load the determined general large model; calculate the loss based on the cross entropy loss function; based on the result of the loss calculation, the gradient descent algorithm adjusts the weight of the model through the Adam optimizer to reduce the loss value; after completing the number of iterations, output the fine-tuned large model.

[0053] In a specific implementation, the goal is to fine-tune a pre-trained large model (such as BERT) through 6 sets of question-answer pairs to improve its performance on the question-answering task. The specific fine-tuning steps are as follows: There are 6 sets of question-answer pairs that constitute the fine-tuning dataset: Question: "What is the central question in economics?" Answer: "The allocation of resources." Question: "What does the law of diminishing marginal utility refer to?" Answer: "As the consumption increases, the satisfaction brought by each additional unit of goods gradually decreases." Question: "What is the definition of art?" Answer: "Art is to create images with aesthetic value through the reflection and processing of real life to meet people's spiritual needs." Question: "In management, what is feedforward control?" Answer: "Feedforward control, also known as ex ante control or pre-control, refers to the organization's prediction and estimation of possible deviations in the work before the work activities officially begin, and taking preventive measures to eliminate possible deviations before they occur." Question: "What is quantum mechanics?" Answer: "Quantum mechanics is a fundamental theory in physics that describes natural phenomena at the smallest energy scales of atoms and subatomic particles." Question: "What is the definition of education?" Answer: "Education refers to a planned, purposeful and organized social activity that promotes the all-round development of individual students by imparting knowledge, cultivating abilities and shaping moral character." The training parameters include: batch size: 2; number of iterations: 3; learning rate: 5e-5 (a low learning rate commonly used when fine-tuning); optimizer: AdamW; loss function: cross entropy loss function, which is used to evaluate the difference between the model output and the correct answer.

[0054] Fine-tuning steps: Step 1: Prepare the model and data.

[0055] Loading a pre-trained model: Start with a large model that has already been pre-trained (such as BERT) and load the pre-trained weights.

[0056] Data preprocessing: Encode the 6 sets of question-answer pairs to fit the model’s input format. Questions and answers will be converted into an input form acceptable to the model (usually vectorized or tokenized).

[0057] Step 2: First iteration.

[0058] Batch 1 input: The model first receives the first two sets of question-answer pairs.

[0059] Forward propagation: Based on the input question and answer, the model predicts what it thinks is the correct answer and outputs a probability distribution.

[0060] Calculating loss: The loss function (cross entropy loss) is used to compare the model's predicted output with the actual answer. Here the loss value reflects the difference between the model and the correct answer, and the larger the value, the less accurate the model's prediction.

[0061] Backpropagation and weight updating: Based on the results of the loss calculation, the gradient descent algorithm adjusts the weights of the model through the Adam optimizer to reduce the loss value. This is the process of the model "learning", that is, it gradually adjusts itself so that it is closer to the correct answer next time.

[0062] Batch 2 input: The model then processes the next two sets of question-answer pairs.

[0063] The steps repeat: the model predicts the answer again, calculates the loss, does backpropagation, and updates the weights.

[0064] Batch 3 input: The last batch of data. Process the last two sets of question-answer pairs, calculate the loss, and update the model weights.

[0065] In the first iteration, the model processed all 6 sets of question-answer pairs (divided into 3 batches of 2 pairs each), and the weight updates in each batch helped the model learn how to better answer these questions.

[0066] Step 3: Second round of iteration. The second round of iteration further optimizes the model's performance and continues to reduce the loss value by letting the model repeatedly process the same question-answer pairs.

[0067] Step 4: The third iteration.

[0068] After 3 iterations of training, the fine-tuned model has learned enough information from the 6 sets of question-answer pairs to accurately answer these specific questions. At this point, the model weights have been optimized, and the fine-tuned model performs better on question-answering tasks than the unfine-tuned pre-trained model.

[0069] The fine-tuned model verification subsystem is used to evaluate the performance of the fine-tuned large model based on the verification data, and determine whether the fine-tuned large model needs to be optimized based on the evaluation results: if optimization is not required, the verified fine-tuned large model is output; if optimization is required, the initial parameters of the determined general large model are reset and transmitted to the model fine-tuning subsystem, which re-fine-tunes based on the training data obtained in the humanities and social sciences. The fine-tuned model is evaluated for performance using specialized verification data. Based on the evaluation results, the system decides whether the model needs to be further optimized, such as adjusting the initial parameters and re-fine-tuning, to achieve higher accuracy and efficiency.

[0070] Input the verification questions in the verification data one by one into the fine-tuned large model to obtain the output answer set; analyze the answer set based on the verification question-verification answer pair to obtain the verification result data, which includes the verification result accuracy, the correct verification result completeness rate and the correct verification result non-void rate; obtain the hardware system operation data during the verification process, which includes the number of hardware system crashes during the verification process, the energy consumption during the verification process and the memory usage during the verification process; obtain the verification result allowable deviation data, which includes the allowable deviation value of the correct verification result completeness rate and the allowable deviation value of the correct verification result non-void rate; obtain the verification requirement performance data stored in the database, which includes the verification result required accuracy, the correct verification result required completeness rate, the correct verification result required non-void rate, the required energy consumption during the verification process and the required memory usage during the verification process.

[0071] By combining the model's answer accuracy, completeness, non-void rate, and hardware performance (energy consumption and memory usage), this evaluation framework comprehensively considers the model's performance in the actual operating environment and ensures the model's practicality and reliability. Energy consumption is related to the actual deployment cost and sustainability of the model, and is paralleled by performance indicators (such as accuracy and completeness) to ensure that the model uses resources within a reasonable range while meeting performance requirements. Memory usage is related to energy consumption, and the two together reflect the efficiency and adaptability of the model on physical resources.

[0072] These parameters together form a comprehensive evaluation system that considers both the model's ability to solve problems in theory (through accuracy, completeness, and non-voidity), and the model's resource efficiency and stability in actual deployment (through energy consumption and memory usage). This multi-dimensional evaluation ensures that the model is efficient, stable, and cost-effective in actual applications, while providing a way to balance model performance and resource consumption.

[0073] Determine whether the number of hardware system crashes during the verification process is 0: if not 0, the evaluation result of the performance of the large model after fine-tuning is that it needs to be optimized; if not 0, obtain the verification index based on the verification result data, hardware system operation data, verification result allowable deviation data and verification requirement performance data.

[0074] ; Where Y is the verification index, f 1 is the first transfer function, f 2 is the second transfer function, zq is the accuracy rate of the verification result, zw is the completeness rate of the correct verification result, zf is the non-void rate of the correct verification result, zn is the energy consumption during the verification process, and zN is the memory usage during the verification process. To correctly verify the allowable deviation value of the result completeness rate, is the allowable deviation value of the non-void rate of the correct verification result, cq is the required correct rate of the verification result, sw is the required completeness rate of the correct verification result, sf is the required non-void rate of the correct verification result, cn is the required energy consumption during the verification process, and cN is the required memory usage during the verification process.

[0075] f 1 The model's correct response rate, completeness rate, and non-void rate are partially evaluated relative to the required deviation. This comparison measures the relative difference in performance by dividing the deviation of the actual value from the target value by the allowed deviation, which helps to quantitatively analyze the quality of the model's answers. 1 Absolute and deviation values ​​are used in the process to quantify the difference between actual performance and expected targets, and even small deviations are identified and taken into account. 2 Paying attention to the comparison of energy consumption and memory usage with the preset standards directly reflects the resource efficiency of the model. The final verification index is determined by combining the outputs of the two transit functions. If the verification index exceeds the predetermined threshold, it means that the performance of the model does not meet the requirements and needs further optimization.

[0076] If the verification index is greater than the verification threshold stored in the database, the evaluation result of the performance of the large model after fine-tuning is that optimization is required; if the verification index is not greater than the verification threshold stored in the database, the evaluation result of the performance of the large model after fine-tuning is that optimization is not required.

[0077] Obtain a first difficulty answer performance index, a second difficulty answer performance index and a third difficulty answer performance index and record them as comparison data; obtain each matching data stored in a database, the matching data including a first difficulty answer performance index matching value, a second difficulty answer performance index matching value and a third difficulty answer performance index matching value; compare the comparison data with the matching data one by one to determine the matching data that is most similar to the comparison data (the similarity value can be calculated using the Euclidean distance); obtain the verification result allowable deviation data stored in the database corresponding to the most similar matching data.

[0078] In practical applications, model outputs may fluctuate due to the diversity of input data and environmental changes. Setting the allowed deviation can provide some flexibility to ensure that the model is still considered to meet the requirements in the face of unforeseen data or environmental changes. No deviation limit may lead to over-optimization of certain data points or over-adjustment of the model to meet very strict standards, thereby ignoring the generalization ability of the model. Allowing deviation ensures that the model optimization remains within a reasonable range and promotes the model to perform well on unseen data.

[0079] The response performance index of the first, second, and third difficulty levels provides actual performance data of the model at different difficulty levels. Adjusting the allowed deviation based on this data ensures that the model has appropriate expected standards for questions of different difficulty levels. By analyzing the performance index of different difficulty levels, the expected performance of the model at each level can be calibrated more finely. For example, if a model performs well on high-difficulty questions, smaller performance fluctuations may be allowed on low-difficulty questions. The response performance index covers multiple aspects such as accuracy, completeness, and non-voidness. By comprehensively evaluating the performance of the model through these indices, we can have a more comprehensive understanding of the advantages and disadvantages of the model, and set a reasonable deviation range accordingly, aiming to optimize the overall performance of the model rather than a single indicator.

[0080] This implementation also includes: An electronic device includes: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the large model fine-tuning training system as described above. A computer-readable storage medium is used to store a program, and when the program is executed by the processor, the large model fine-tuning training system as described above is implemented.

Claims

1. Large model fine-tuning training system, characterized by: It includes data acquisition subsystem, model fine-tuning subsystem and post-fine-tuning model verification subsystem, among which: The data acquisition subsystem is used to acquire training data and verification data in the field of humanities and social sciences. The training data includes a number of training question-training answer pairs, and the verification data includes a number of verification question-verification answer pairs. The model fine-tuning subsystem is used to determine the general large model to be fine-tuned, and train and fine-tune the determined general large model based on training data and small sample learning, and output the fine-tuned large model for the humanities and social sciences; The fine-tuned model verification subsystem is used to evaluate the performance of the fine-tuned large model based on the verification data, and determine whether the fine-tuned large model needs to be optimized based on the evaluation results: If optimization is not required, the fine-tuned large model after verification is output; If optimization is required, the initial parameters of the determined general large model are reset and transmitted to the model fine-tuning subsystem, which then re-fine-tunes the model based on the training data obtained in the humanities and social sciences.

2. The large model fine-tuning training system according to claim 1, characterized in that: The step of obtaining training data comprises the following steps: Obtain data to be screened related to the content in the humanities and social sciences, including questions to be screened and answers to the questions to be screened. The content in the humanities and social sciences includes professional textbooks, academic papers and internship training cases related to economics, management, literature, law, science, engineering, education and art; Randomly grouping the data to be screened to obtain several data groups to be screened; Performing intra-group screening on each data group to be screened to obtain a screened data group; Determine the most similar post-screening data group pair, perform secondary screening on the post-screening data group pair, and obtain a secondary post-screening data group; The data group after secondary screening and the data group after screening without secondary screening are split to obtain training data.

3. The large model fine-tuning training system according to claim 2 is characterized in that: The method of obtaining the screened data set comprises the following steps: Randomly select any one of the questions to be screened in the data group to be screened, obtain the text keyword combination to be screened of the question to be screened and save it to the text keyword set corresponding to the data group to be screened; Obtain the text keywords to be screened of the next question to be screened in the data group to be screened one by one and record them as the text keyword combination to be compared; Perform similarity comparison between the text keyword combination to be compared and the text keyword combination to be screened in the text keyword set one by one: If the text keyword combination to be compared is similar to any text keyword combination to be screened in the text keyword set, then the text keyword combination to be compared is deleted and the question to be screened corresponding to the text keyword combination to be compared and the answer to the question to be screened are deleted; If the text keyword combination to be compared is not similar to any text keyword combination to be screened in the text keyword set, the text keyword combination to be compared is saved in the text keyword set, and the question to be screened corresponding to the text keyword combination to be compared and the answer to the question to be screened are retained; The data to be screened after the in-group screening is recorded as a screened data group, and the text keyword set corresponding to the screened data group is saved.

4. The large model fine-tuning training system according to claim 3 is characterized in that: The method of determining the most similar screened data group pair and performing secondary screening on the screened data group pair to obtain a secondary screened data group comprises the following steps: Perform similarity judgment on the text keyword sets corresponding to any two filtered data groups: If two text keyword sets are not similar, the corresponding two filtered data sets will not be screened again; If two text keyword sets are similar, then the corresponding two filtered data groups are the most similar filtered data group pairs; determining the number of to-be-screened questions in two screened data sets in the most similar pair of screened data sets; The large number of filtered data sets is used as the benchmark, and the questions to be filtered in the small number of filtered data sets are compared with the questions to be filtered in the large number of filtered data sets for similarity: If they are similar, delete the unscreened questions in the screened data group with a small number; If they are not similar, the unscreened problems in the screened data group with a small number are retained; Output the two filtered data sets to obtain a secondary filtered data set.

5. The large model fine-tuning training system according to claim 1, characterized in that: The step of determining the general large model to be fine-tuned comprises the following steps: The training data is grouped according to difficulty levels to obtain first difficulty training data, second difficulty training data, and third difficulty training data; Input the training questions in the first difficulty training data, the second difficulty training data and the third difficulty training data into each candidate universal large model one by one, and obtain the answer evaluation data of each candidate universal large model, the answer evaluation data including the first difficulty answer evaluation data, the second difficulty answer evaluation data and the third difficulty answer evaluation data; Determine the general large model to be fine-tuned based on the response evaluation data.

6. The large model fine-tuning training system according to claim 5, characterized in that: The step of grouping the training data by difficulty level comprises the following steps: Obtain training question keyword combinations corresponding to each training question in the training data; Obtain the keyword combination sets of each difficulty level stored in the database, and determine whether the training question keyword combination hits one of the keyword combination sets of each difficulty level: If there is a hit, then the difficulty coefficient stored in the database corresponding to the hit difficulty level keyword combination set is obtained; If there is no hit, the difficulty coefficient corresponding to the keyword combination of the training question is 0; Get the correct rate of human responses to each training question; Determine the difficulty distribution coefficient for each training question based on the difficulty coefficient and the correct answer rate: ； In the formula, A is the difficulty distribution coefficient, Nx is the difficulty coefficient, DL is the correct rate of human response, and a is The weight factor of Nx stored in the database is The weight factor of DL stored in the database; If the difficulty allocation coefficient is less than the first difficulty allocation threshold stored in the database, the training question-training answer pair corresponding to the difficulty allocation coefficient is the first difficulty training data; If the difficulty allocation coefficient is not less than the first difficulty allocation threshold stored in the database, but less than the second difficulty allocation threshold stored in the database, then the training question-training answer pair corresponding to the difficulty allocation coefficient is the second difficulty training data; If the difficulty allocation coefficient is not less than the second difficulty allocation threshold stored in the database, the training question-training answer pair corresponding to the difficulty allocation coefficient is the third difficulty training data.

7. The large model fine-tuning training system according to claim 5, characterized in that: The method of determining the general large model to be fine-tuned based on the response evaluation data comprises the following steps: Obtaining first difficulty response parameter data, second difficulty response parameter data, and third difficulty response evaluation data stored in a database; Based on the first difficulty answer evaluation data and the first difficulty answer parameter data, the first difficulty answer performance index corresponding to the candidate general large model is obtained. The first difficulty answer parameter data includes the first difficulty answer accuracy rate, the first difficulty correct answer completeness rate, the first difficulty correct answer reaction time and the first difficulty correct answer non-empty rate. The first difficulty answer parameter data includes the first difficulty parameter answer accuracy rate, the first difficulty parameter correct answer completeness rate, the first difficulty parameter correct answer reaction time and the first difficulty parameter correct answer non-empty rate; Based on the second difficulty answer evaluation data and the second difficulty answer parameter data, the second difficulty answer performance index corresponding to the candidate general large model is obtained, and the second difficulty answer parameter data includes the second difficulty answer accuracy rate, the second difficulty correct answer completeness rate, the second difficulty correct answer reaction time, and the second difficulty correct answer non-empty rate. The second difficulty answer parameter data includes the second difficulty parameter answer accuracy rate, the second difficulty parameter correct answer completeness rate, the second difficulty parameter correct answer reaction time, and the second difficulty parameter correct answer non-empty rate; Based on the third difficulty response evaluation data and the third difficulty response parameter data, the third difficulty response performance index corresponding to the candidate general large model is obtained. The third difficulty response parameter data includes the third difficulty response accuracy rate, the third difficulty correct answer completeness rate, the third difficulty correct answer reaction time and the third difficulty correct answer non-empty rate. The third difficulty response parameter data includes the third difficulty parameterized answer accuracy rate, the third difficulty parameterized correct answer completeness rate, the third difficulty parameterized correct answer reaction time and the third difficulty parameterized correct answer non-empty rate; The first difficulty response performance index, the second difficulty response performance index and the third difficulty response performance index are weighted and summed to obtain the overall performance effect value of the large model; The candidate general large model corresponding to the smallest overall performance effect value of the large model is determined as the general large model to be fine-tuned.

8. The large model fine-tuning training system according to claim 7, characterized in that: The method of training and fine-tuning the determined general large model based on the training data and small sample learning, and outputting the fine-tuned large model, includes the following steps: Initialize the fine-tuning parameters of the determined general large model, including the learning rate, the batch size of the input training data, and the number of iterations; Create task prompts and construct model inputs; Load the determined general large model; Calculate the loss based on the cross entropy loss function; Based on the results of the loss calculation, the gradient descent algorithm adjusts the weights of the model through the Adam optimizer to reduce the loss value; After completing the number of iterations, the fine-tuned large model is output.

9. The large model fine-tuning training system according to claim 7, characterized in that: The performance evaluation of the fine-tuned large model based on the verification data includes the following steps: Input the verification questions in the verification data into the fine-tuned large model one by one to obtain the output answer set; Based on the verification question-verification answer pair, the answer set is analyzed to obtain verification result data, which includes the verification result correctness rate, the correct verification result completeness rate and the correct verification result non-void rate; Obtaining hardware system operation data during the verification process, the hardware system operation data including the number of hardware system crashes during the verification process, the energy consumption during the verification process, and the memory usage during the verification process; Acquire verification result allowable deviation data, the verification result allowable deviation data including a correct verification result completeness rate allowable deviation value and a correct verification result non-void rate allowable deviation value; Acquire the verification requirement performance data stored in the database, the verification requirement performance data including the required accuracy rate of the verification result, the required completeness rate of the correct verification result, the required non-void rate of the correct verification result, the required energy consumption during the verification process, and the required memory usage during the verification process; Determine whether the number of hardware system crashes during the verification process is 0: If it is not 0, the evaluation result of the performance of the large model after fine-tuning is that it needs to be optimized; If it is not 0, the verification index is obtained based on the verification result data, the hardware system operation data, the verification result allowable deviation data and the verification requirement performance data; If the validation index is greater than the validation threshold stored in the database, the evaluation result of the performance of the large model after fine-tuning is that it needs to be optimized; If the validation index is not greater than the validation threshold stored in the database, the evaluation result of the performance of the large model after fine-tuning is that no optimization is required.

10. The large model fine-tuning training system according to claim 9, characterized in that: The method of obtaining the verification result allowable deviation data comprises the following steps: Obtaining a first difficulty response performance index, a second difficulty response performance index, and a third difficulty response performance index and recording them as comparison data; Acquire each matching data stored in the database, the matching data including a first difficulty answer performance index matching value, a second difficulty answer performance index matching value, and a third difficulty answer performance index matching value; Compare the comparison data with the matching data one by one to determine the matching data that is most similar to the comparison data; The verification result allowable deviation data corresponding to the most similar matching data stored in the database is obtained.

Images