Training method, scoring method and device of a scoring model

Abstract

The application provides a scoring model training method, a scoring method and a device, wherein the training method comprises the following steps: obtaining pre-training data and training data, the pre-training data comprising a first sample test question, a reference answer corresponding to the first sample test question, a first user answer and an answer score; the training data comprising a scoring specification, a second sample test question and a second user answer corresponding to the second sample test question; training an initial model based on the pre-training data to obtain a pre-training model; scoring the second user answer by using the pre-training model based on the second sample test question and the scoring specification to obtain a predicted score of the second user answer and scoring analysis, wherein the scoring analysis is used to explain the composition of the predicted score; the method overcomes the defects that the current scoring scheme is difficult to accurately understand the text semantics, resulting in low scoring accuracy and only having the overall score, realizes automatic scoring of the user answer, guarantees the scoring accuracy and the scoring efficiency, and provides data support for subsequent consolidation learning.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a training method, scoring method, and apparatus for a scoring model. Background Technology

[0002] In the educational process, testing and assessment have become an indispensable part of daily teaching. Automated scoring systems can greatly reduce teachers' workload, improve the fairness and efficiency of scoring, and thus contribute to the development of education and teaching.

[0003] Currently, automatic scoring systems primarily rely on natural language processing techniques, such as bag-of-words models, topic models, word vectors, and neural networks. These systems transform textual information into mathematical forms, extract key information, and then score the text according to predefined rules. However, due to the complexity of natural language, they often struggle to fully and accurately understand the semantics of the text, resulting in relatively low scoring accuracy. Furthermore, their output is only an overall score, which is inconvenient for users to further reinforce their learning. Summary of the Invention

[0004] This invention provides a training method, scoring method, and apparatus for a scoring model, which addresses the shortcomings of existing technologies, such as low scoring accuracy and the fact that only an overall score is obtained after scoring, which is not conducive to further learning. The trained scoring model can accurately understand the semantics of the input text, realize automatic scoring of user answers, and provide score analysis, ensuring the accuracy of the scoring and providing data support for users to further consolidate their learning.

[0005] This invention provides a method for training a scoring model, comprising:

[0006] Acquire pre-training data and training data. The pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score for the first sample test question. The training data includes scoring specifications, a second sample test question, and its corresponding second user answer.

[0007] Based on the pre-training data, the initial model is trained to obtain the pre-trained model;

[0008] Based on the second sample test questions and the scoring criteria, the pre-trained model is applied to score the second user's answer to obtain the predicted score and score analysis of the second user's answer. The score analysis is used to explain the composition of the predicted score.

[0009] Based on the training data, the predicted score, and the score analysis, the pre-trained model is trained to obtain the scoring model.

[0010] According to a training method for a scoring model provided by the present invention, the step of applying the pre-trained model to score the second user's answer based on the second sample test questions and the scoring criteria, to obtain a predicted score and score analysis for the second user's answer, includes:

[0011] Based on the second sample test questions, the pre-trained model is used to generate answers, and the predicted answers corresponding to the second sample test questions are obtained.

[0012] Based on the predicted answer and the scoring criteria, the pre-trained model is used to score the second user's answer, resulting in a predicted score and a score analysis.

[0013] According to a training method for a scoring model provided by the present invention, the step of applying the pre-trained model to score the second user's answer based on the predicted answer and the scoring specification to obtain a predicted score and a score analysis includes:

[0014] Based on the scoring criteria, the pre-trained model is used to decompose the predicted answer into scoring points to obtain the scoring point analysis;

[0015] Based on the score point analysis, the pre-trained model is applied to score the second user's answer, resulting in a predicted score and a score analysis.

[0016] According to a training method for a scoring model provided by the present invention, the step of training an initial model based on the pre-training data to obtain a pre-trained model includes:

[0017] Based on the pre-trained data, the initial model is applied to predict character units one by one to obtain the prediction results corresponding to the pre-trained data.

[0018] Based on the prediction results and the pre-training data, the initial model is trained to obtain a pre-trained model, which is built on the basis of a large language model.

[0019] According to a training method for a scoring model provided by the present invention, the step of training the initial model based on the prediction result and the pre-training data to obtain a pre-trained model includes:

[0020] Based on the prediction results and the pre-training data, determine the prediction loss;

[0021] Based on each predicted number in the prediction results and each number in the pre-training data, determine the digital loss;

[0022] Based on the prediction loss and the digital loss, the initial model is trained to obtain a pre-trained model.

[0023] The present invention also provides a scoring method, comprising:

[0024] Determine the user's answer to be scored, and the corresponding question for that answer;

[0025] Based on the test question, a scoring model is applied to score the user's answer, resulting in an answer score and a score analysis; the score analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any of the above.

[0026] According to a scoring method provided by the present invention, the scoring analysis includes the error score points corresponding to the answer score and the reasons for the errors;

[0027] The process of applying a scoring model to score the user's answer based on the test question, resulting in an answer score and score analysis, includes:

[0028] Based on the test questions, the user's answers, and the error types, a scoring prompt statement is generated;

[0029] The scoring prompt statement is input into the scoring model to obtain the answer score output by the scoring model, as well as the error type corresponding to the error point and the error reason.

[0030] The present invention also provides a training apparatus for a scoring model, comprising:

[0031] The acquisition unit is used to acquire pre-training data and training data. The pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score for the first sample test question. The training data includes a scoring specification, a second sample test question, and its corresponding second user answer.

[0032] A pre-training unit is used to train an initial model based on the pre-training data to obtain a pre-trained model;

[0033] The prediction unit is used to score the second user's answer based on the second sample test questions and the scoring criteria, using the pre-trained model to obtain the predicted score and score analysis of the second user's answer, wherein the score analysis is used to explain the composition of the predicted score.

[0034] A training unit is used to train the pre-trained model based on the training data, the predicted score, and the score analysis to obtain a scoring model.

[0035] The present invention also provides a scoring device, comprising:

[0036] A determining unit is used to determine the user's answer to be scored, and the test question corresponding to the user's answer;

[0037] A scoring unit is used to score the user's answer based on the test question using a scoring model, thereby obtaining an answer score and a score analysis; the score analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any of the above.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a training method or a scoring method for a scoring model as described above.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method or a scoring method for the scoring model as described above.

[0040] The training method, scoring method, and apparatus for the scoring model provided by this invention improve the model's ability to understand input text through pre-training, enabling accurate understanding of the semantics of the input text and thus accurate answering and scoring. Based on this, fine-tuning of the scoring is performed to better adapt the model to the question-answering task. It can output predicted scores and score analyses based on the input questions, scoring specifications, and user answers, explaining the composition of the predicted scores and providing a basis for their delivery. This overcomes the shortcomings of traditional automatic scoring schemes, such as the difficulty in comprehensively and accurately understanding the semantics of the text, resulting in low scoring accuracy, and the fact that only an overall score is obtained after scoring, which is not conducive to further learning. This invention achieves automatic scoring of user answers and provides score analyses, ensuring scoring accuracy and efficiency while providing data support for subsequent consolidation learning. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the training method of the scoring model provided by the present invention;

[0043] Figure 2 This is an example diagram of the pre-training data provided by the present invention;

[0044] Figure 3 This is an example diagram of the pre-training process provided by the present invention;

[0045] Figure 4This is a flowchart illustrating the scoring method provided by the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of the training device for the scoring model provided by the present invention;

[0047] Figure 6 This is a schematic diagram of the scoring device provided by the present invention;

[0048] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] Currently, the main technologies for automatically scoring user answers in online education scenarios include:

[0051] Firstly, methods based on the bag-of-words model and topic modeling model model the matching between the standard answer and the user's answer for each question. The user's answer with a high match with the standard answer will receive a higher score. This approach only matches at the grammatical level of the text and cannot model the syntactic and semantic features of the standard answer and the user's answer. In this case, if the user's answer has the same meaning as the standard answer but is expressed differently, it is very likely to be given a low score; conversely, if the user's answer has similarities in expression to the standard answer but is completely different in meaning, it is very likely to be given a high score.

[0052] Secondly, there is the method based on word vectors and neural networks. This method models the test information and user responses across all test questions. This approach takes into account the grammatical level of the text and can avoid the aforementioned problems to some extent.

[0053] Third, in order to adapt to different scenarios, scenario features can be added on the basis of the second method, so that the model can learn to quickly adapt to the scoring task of a specific scenario on a small amount of scenario data.

[0054] However, each of the above solutions has certain drawbacks, namely:

[0055] First, methods based on bag-of-words and topic models only model the grammar of user responses. Therefore, they cannot accurately and effectively score user responses with the same semantics but different grammars, or the same grammar but different semantics, which has a great drawback in scoring.

[0056] Secondly, while word vector and neural network-based methods can avoid some of the shortcomings of the first approach, they still have three serious problems. First, they require a large amount of scoring data for modeling. Second, scoring requirements vary across different scenarios, making them unsuitable for specific scenarios. For example, the rigor of scoring standards differs in different exam scenarios, and a simple neural network model cannot model these standards. Third, they cannot handle the problem of multiple solutions to a single question, where there are multiple ways to solve a question and the user's answer differs from the standard answer. Neural network-based models lack the ability to solve problems and therefore cannot score user answers at the semantic level.

[0057] Third, while neural network-based and adaptive methods can alleviate the first two problems of the second approach by adding a small amount of data in specific scenarios, the third problem remains unresolved. Furthermore, all the above approaches are end-to-end automatic scoring schemes. When scoring user responses, they only output an overall score, without providing explanations or justifications for the given score. This is extremely inconvenient for users' subsequent learning.

[0058] To address this, the present invention provides a training method for a scoring model, aiming to improve the model's ability to understand input text through pre-training, so as to accurately understand the semantics of the input text and thus be able to answer questions and score accurately. On this basis, the scoring is fine-tuned so that the model can better adapt to the question-answering task. It can output predicted scores according to the input questions, scoring specifications and user answers, and can also provide scoring parsing to explain the user's answer. While ensuring scoring accuracy and efficiency, it can also provide assistance for subsequent consolidation learning. Figure 1 This is a flowchart illustrating the training method of the scoring model provided by the present invention, as shown below. Figure 1 As shown, this method can be applied to automatic scoring systems across all learning stages and subjects. It is an interpretable scoring method based on question comprehension and student response comprehension, and includes:

[0059] Step 110: Obtain pre-training data and training data. The pre-training data includes the first sample test questions, as well as the corresponding reference answers, first user answers, and answer scores for the first sample test questions. The training data includes the scoring specifications, the second sample test questions, and their corresponding second user answers.

[0060] Specifically, before training the model, training data needs to be acquired. In this embodiment of the invention, to ensure the model's generalization performance and accuracy in the scoring task, the entire training process can be divided into two stages: a pre-training stage and a training stage. The pre-training stage improves the model's ability to understand the input text, enabling it to accurately understand the semantics of the input text. This allows it to accurately predict the text at the next moment based on the input text, thus possessing the ability to answer questions accurately and score accurately. On this basis, fine-tuning training is performed to fine-tune the model's parameters to better suit the scoring task and provide predicted scores and score analysis for user answers. This improves the accuracy of scoring without affecting the model's generalization performance and provides a reference for subsequent learning.

[0061] Based on this, in this embodiment of the invention, it is first necessary to obtain training samples for each stage, i.e., sample data for the pre-training stage (hereinafter referred to as pre-training data), and sample data for the training stage (hereinafter referred to as training data). Since the purpose of the pre-training stage is to improve the model's ability to understand text, enabling it to accurately understand the semantics of the input text and accurately predict the text at the next moment based on the input text, the model trained in this way can perform accurate question answering and scoring, thus laying the foundation for subsequent scoring tasks. Therefore, the sample data for the pre-training stage must contain data that enables the model to learn in the aforementioned multiple aspects, understand different inputs, and thus perform accurate question answering and scoring. For example, it may include test questions, answers, scores, etc.

[0062] Specifically, the pre-training data includes sample questions, answers, and answer scores. For ease of distinction, the sample questions here can be referred to as the first sample questions, and the answers can include the standard answer and the user's answer for the first sample questions, referred to as the reference answer and the first user's answer, respectively. In short, the pre-training data includes the first sample questions, as well as the corresponding reference answer, first user's answer, and answer score.

[0063] Figure 2 This is an example diagram of the pre-training data provided by the present invention, such as... Figure 2 As shown, the pre-training dataset D includes n pre-training data points, which can be represented as:

[0064] D = (data0, data1, ..., data i …,data n )

[0065] Each pre-training dataset contains a first sample question, a corresponding answer, and a first user's answer. answerAnd the answer score, which can be represented as:

[0066]

[0067] In the formula, data i Let 'question' represent the i-th pre-training data in the pre-training dataset D. i Let answer represent the first sample question in the i-th pre-training data. i Then it is a question i The corresponding reference answer, For question i The corresponding first user answer, score i for The answer will be graded.

[0068] Correspondingly, the sample data during the training phase includes data that enables the model to be better suited for scoring tasks, allowing it to answer, score, and parse questions based on the input text. For example, it may include test questions, corresponding user answers, and scoring criteria for those answers. In other words, the training data includes second sample test questions, second user answers corresponding to those questions, and scoring criteria.

[0069] It is worth noting here that the first sample test questions in the pre-training phase and the second sample test questions in the training phase can be the same or different. If the first sample test questions and the second sample test questions are the same, the first user's answer and the second user's answer can be the same or different. They can choose accordingly based on the actual situation. This embodiment of the invention does not make specific limitations on this.

[0070] In the training samples at each stage, the sample test questions (first sample test questions and second sample test questions) can be obtained by manual input, or by image acquisition devices to capture images of existing paper test questions and extracting text from the captured sample test question images. They can also be obtained through web search, web crawling, or by downloading / capturing / searching directly from test question banks or specific test question websites. This embodiment of the invention does not specifically limit these methods.

[0071] The reference answers can be obtained by experts or teachers with rich experience, or they can be obtained along with the first sample test questions. That is, the test questions and their corresponding standard answers can be obtained directly from the test question bank or specific website that carries the standard answers, and used as the first sample test questions and their corresponding reference answers. Alternatively, the answers can be extracted from paper materials such as textbooks and teaching aids that contain test question answers. This embodiment of the invention does not make specific limitations on this.

[0072] The user answers (first user answer and second user answer) can be obtained by capturing images of users answering sample questions using an image acquisition device and extracting the answers, or they can be obtained together with the sample questions. That is, the questions and user answers can be directly obtained from a question bank or a specific website that carries user answers, serving as sample questions and their corresponding user answers. This embodiment of the invention does not make specific limitations on this.

[0073] Answer scores can be obtained through review by experienced experts or teachers, or they can be obtained along with the first sample test questions and the first user answers. That is, test questions, user answers, and answer scores can be obtained directly from a question bank or test paper bank with user answer scores as the first sample test questions, along with their corresponding first user answers and answer scores. Alternatively, they can be obtained by scoring and identifying paper materials with answer scores, such as practice questions and answer papers. This embodiment of the invention does not specifically limit this.

[0074] The scoring criteria can be formulated by experienced experts or teachers, or they can be derived from the scoring of test papers and exercises based on user responses and answers. Alternatively, they can be determined based on currently used scoring criteria. This embodiment of the invention does not impose any specific limitations on these criteria.

[0075] Step 120: Train the initial model based on the pre-training data to obtain the pre-trained model;

[0076] Specifically, after obtaining the pre-training data in step 110, the model can be pre-trained based on the pre-training data to obtain the pre-trained model, i.e., the pre-trained model.

[0077] It is worth noting that before using pre-training data for model pre-training, it is necessary to first determine the initial model. Considering that in traditional scoring methods based on natural language processing technology, the complexity of natural language often makes it difficult to fully and accurately understand the semantics of the text, resulting in low scoring accuracy, in this embodiment of the invention, in order to achieve accurate understanding of the input text, a general large model with good understanding of natural language can be used when constructing the initial model. This model can be used to understand the text and its semantic information, thereby accurately predicting the text at the next moment and achieving accurate scoring.

[0078] Here, the general-purpose large-scale model refers to a large-scale language model trained on a large amount of data in a general domain, with a huge number of parameters and basic dialogue logic. For example, the Spark Cognitive Large-Scale Model has good capabilities in various domains, such as grammar correction, solving simple math problems, and answering basic knowledge questions. However, when applied to specific downstream tasks, such as difficulty assessment and error analysis of test answers, further training is usually required to improve its performance on downstream tasks.

[0079] Based on this, in this embodiment of the invention, after the initial model is constructed, in order to improve the performance of the model on the scoring task and enable it to have expert-level capabilities in automatic scoring, it is also necessary to perform specific corpus pre-training and fine-tuning training, so that the model can have extremely strong scoring capabilities in the scoring domain while retaining its original natural and fluent human-computer dialogue capabilities.

[0080] Specifically, after obtaining the pre-training data, the pre-training stage can be entered to pre-train the initial model to improve its understanding ability, thereby obtaining a pre-trained model. Specifically, this can involve using the initial model to make predictions based on the pre-training data. Leveraging the initial model's text continuation ability, it can automatically predict subsequent text based on the preceding context, thus predicting and outputting the corresponding text content, i.e., the prediction result, based on the pre-training data.

[0081] That is, the initial model can be applied to predict the first sample question, reference answer, first user answer, and answer score in the pre-training data to predict the token at the next time step. After the prediction is completed, the prediction result corresponding to the pre-training data output by the model can be obtained. Here, the token can be divided into units of words, characters, or even smaller character units. This embodiment of the invention does not specifically limit this.

[0082] Following this, the initial model can be trained based on the prediction results output by the model to obtain a trained initial model. Specifically, this involves using the model's prediction output to perform self-supervised training on the initial model to obtain a pre-trained model. More specifically, this can involve using the prediction results and pre-training data as a benchmark to determine the difference between the model's predictions and the actual data. Based on this, the model's loss on the prediction task is measured, and the model's parameters are iterated based on the loss. This is done so that the predicted results output by the parameter-adjusted model are as close as possible to, or even identical to, the pre-training data, thus obtaining a trained pre-trained model.

[0083] Step 130: Based on the second sample test questions and scoring criteria, apply the pre-trained model to score the second user's answer, and obtain the predicted score and score analysis of the second user's answer. The score analysis is used to explain the composition of the predicted score.

[0084] Specifically, after obtaining the pre-trained model in step 120, a scoring task can be performed based on the training data using this pre-trained model to determine the predicted score and score analysis corresponding to the training data. Here, the score analysis is used to explain the composition of the predicted score.

[0085] Considering that traditional automatic scoring systems lack an effective error feedback mechanism, outputting only an overall score without providing specific error feedback, they cannot provide a basis or support for the scoring. This not only greatly reduces the reliability and credibility of the scoring, but also leaves subsequent learning without a basis, meaning users cannot know their own answer performance, the reasons for errors, etc., making it impossible to carry out targeted reinforcement learning, resulting in decreased learning efficiency and a poor experience.

[0086] Therefore, to enable the model to analyze user responses during scoring tasks and provide information such as scores, errors, and reasons for errors, thus supporting the output score and facilitating further learning for users, this embodiment of the invention allows for further fine-tuning of the pre-trained model, which possesses strong comprehension and answering capabilities, obtained after the previous pre-training step. This enables the model to not only review the questions and provide reference answers during the scoring task in the fine-tuning training phase, but also review both the reference answers and the user's answers for automatic scoring. The model then outputs predicted scores and score analyses to explain the basis for these predicted scores, improving the credibility of the scores and facilitating targeted learning.

[0087] In detail, after obtaining the training data and the pre-trained model, the training data can be used as input to the pre-trained model so that the pre-trained model can understand it and output its predicted score and score analysis accordingly. Specifically, the pre-trained model first understands the second sample question input, answers the question, and outputs the corresponding reference answer. Then, based on the reference answer, it can score the second user's answer using the given scoring criteria, thereby obtaining the predicted score and score analysis output by the model.

[0088] In other words, based on the reference answer and scoring criteria, a pre-trained model can be applied to analyze the second user's answer to determine the correct and incorrect parts. The score can then be determined by comparing the answer to the scoring criteria. Errors can be analyzed to identify the causes and corresponding remedial measures. Finally, the model outputs a predicted score for the second user's answer, along with a score analysis. This score analysis can include the user's answer score distribution, such as which points were scored, the differences in scores across different sections, the user's errors (such as which points were missed, the errors in different sections), the causes of the errors, and remedial measures.

[0089] Here, the scoring criteria can be pre-defined / acquired / summarized scoring standards for user answers. For example, for a comprehensive application question (total 10 points), the criteria could be: 2 points for the first sub-question (lower difficulty), 3 points for the second sub-question (medium difficulty), and 5 points for the third sub-question (higher difficulty). Similarly, for an essay question (total 20 points), the criteria could be: 5 points for a correct argument, 6 points for correct evidence, 7 points for a correct reasoning process, and 2 points for the final summary.

[0090] Step 140: Based on the training data, as well as the predicted scores and score parsing, train the pre-trained model to obtain the scoring model.

[0091] Specifically, after obtaining the predicted score and score analysis output by the pre-trained model through the above process, the predicted score and score analysis, as well as the training data, can be used to train the pre-trained model to obtain the trained score model.

[0092] Understandably, after completing the above scoring task and obtaining the predicted scores and score parsing of the pre-trained model for the training data output, the model's loss in the above scoring task can be measured using the model's prediction results and the training data. Based on the loss, the model parameters can be fine-tuned to finally obtain the trained scoring model.

[0093] Specifically, after the pre-trained model outputs predicted scores and score analyses, it can be used for self-supervised fine-tuning. This involves using the training data and the predicted scores and score analyses to identify any deficiencies in the model's predictions for the scoring task. Based on this, the model's loss is estimated, and the model's parameters are fine-tuned to ensure that the predicted scores output by the fine-tuned model correspond as closely as possible to the user's actual responses, and that the score analyses accurately reflect the user's responses and are consistent with the second user's answer. The final result is the trained model, i.e., the scoring model.

[0094] In this embodiment of the invention, fine-tuning is performed on the scoring task based on the pre-trained model, enabling the model to understand and answer the input second sample test questions in a natural language dialogue manner. It can also analyze the second user's answer according to the given scoring criteria, and finally give a predicted score and score analysis. Based on this, the model is trained, and finally a trained scoring model is obtained. The trained scoring model can accurately understand the input text and give specific error feedback, which greatly reduces the need for labeled data, can handle complex problems, and can provide a basis for subsequent consolidation learning.

[0095] Furthermore, it should be noted that the scoring model trained using the aforementioned methods possesses strong natural language processing capabilities, enabling accurate understanding of natural language. Therefore, it can address the issue of low scoring accuracy in traditional solutions due to grammatical and semantic limitations. Additionally, regarding situations with multiple solutions to a single question, the scoring model in this embodiment exhibits human-like comprehension and problem-solving abilities. When a user's answer differs from the reference answer, the model can, to some extent, independently determine the correctness of the user's answer, rather than relying entirely on the standard answer. Moreover, the explanatory scoring analysis output by the model not only supports score prediction but also identifies errors and their reasons, making human-computer interaction more natural and fluid.

[0096] The training method for the scoring model provided by this invention improves the model's understanding of input text through pre-training, enabling accurate understanding of the semantics of the input text and thus accurate answering and scoring. Based on this, fine-tuning of the scoring is performed to better adapt the model to the question-answering task. It can output predicted scores and score analyses based on the input questions, scoring criteria, and user answers, explaining the composition of the predicted scores and providing a basis for their delivery. This overcomes the shortcomings of traditional automatic scoring schemes, such as the difficulty in comprehensively and accurately understanding text semantics, leading to low scoring accuracy, and the fact that only an overall score is obtained, which is not conducive to further learning. This method achieves automatic scoring of user answers and provides score analyses, ensuring scoring accuracy and efficiency while providing data support for subsequent consolidation learning.

[0097] Based on the above embodiments, step 130 includes:

[0098] Based on the second sample questions, a pre-trained model is used to generate answers, and the predicted answers corresponding to the second sample questions are obtained.

[0099] Based on the predicted answer and scoring criteria, a pre-trained model is used to score the second user's answer, resulting in a predicted score and a score analysis.

[0100] Specifically, in step 130, the process of applying a pre-trained model to score the second user's answer based on the second sample test questions and scoring criteria, and obtaining the predicted score and score analysis for the second user's answer, may include:

[0101] First, based on the pre-trained model obtained in the previous step, the pre-trained model can be applied to generate answers based on the training data to obtain the predicted answers corresponding to the second sample questions. In other words, based on the second sample questions, answers can be generated through the pre-trained model so that the pre-trained model can answer the questions according to the second sample questions and output its predicted answers, thereby obtaining the predicted answers to the second sample questions.

[0102] Specifically, this could involve taking the second sample question from the training data as input and feeding it into a pre-trained model so that the pre-trained model can understand it, answer the question, and output its predicted answer. In other words, the pre-trained model can first understand the input second sample question, then answer the question based on the semantic correspondence of the understood question, and output its predicted answer as the predicted answer for the second sample question.

[0103] Subsequently, this predicted answer can be used to score user answers, yielding corresponding scores and score analyses. That is, when scoring based on the predicted answer, the model's predicted answer can be considered a reference answer. Based on the reference answer of the second sample question and the scoring criteria, the pre-trained model is applied to score the second user answer corresponding to the second sample question, thus obtaining the predicted score and score analysis of the second user answer output by the pre-trained model. The score analysis here is used to explain the composition of the predicted score.

[0104] Specifically, this could involve, based on the predicted answers to the second sample questions obtained in the previous step, applying a pre-trained model to analyze the second user's answers according to the given scoring criteria. This analysis would identify correct and incorrect answers, and further analyze the errors to determine their causes and corresponding remedial measures. Ultimately, this would yield the model's predicted score and a score analysis. The score analysis could include the user's answer score distribution, such as which score points were scored, the differences in scores across different sections, the user's errors, such as which score points were missed, the errors in each section, the causes of the errors, and remedial measures.

[0105] Based on the above embodiments, and based on the predicted answer and scoring criteria, a pre-trained model is applied to score the second user's answer, resulting in a predicted score and score analysis, including:

[0106] Based on the scoring criteria, a pre-trained model is used to break down the scoring points of the predicted answer to obtain the scoring point analysis.

[0107] Based on the score point analysis, a pre-trained model is applied to score the second user's answer, resulting in a predicted score and score analysis.

[0108] Specifically, the process of applying a pre-trained model to score the second user's answer based on the predicted answer and scoring criteria, resulting in predicted scores and score parsing, may include:

[0109] After obtaining the predicted answers to the second sample test questions, when scoring them, it is important to consider that current scoring schemes typically only provide an overall score and cannot conduct in-depth analysis of user answers or provide a basis for scoring. This not only reduces the credibility of the scoring but also leaves users without a reference for further learning, resulting in a poor user experience.

[0110] Therefore, in order for the model to analyze the user's response when scoring the second user's answer, and to provide information such as the user's score, errors, and reasons for errors, so as to support the output score and facilitate the user's next learning step, in this embodiment of the invention, after the model predicts the predicted answer to the second sample question, this predicted answer can be regarded as the reference answer to the second sample question. According to the scoring criteria, this reference answer is analyzed to identify the key scoring points in the reference answer, and the second user's answer is scored accordingly. The user's response is analyzed, and the final score and score analysis are given, thus obtaining the predicted score and score analysis of the second user's answer.

[0111] Specifically, after obtaining the predicted answer, this predicted answer is regarded as the reference answer for the second sample test question. When scoring the second user's answer accordingly, the composition of the reference answer can be analyzed according to the scoring criteria to determine the score of each step / argument / key point, thereby obtaining the score point analysis of the reference answer. That is, the score point analysis can be obtained by using the scoring criteria as a benchmark and applying the pre-trained model to decompose the score points of the reference answer, confirming each score point and the score corresponding to each score point.

[0112] It is worth noting here that when the scoring points need to be adjusted in a specific scenario, the scoring point analysis obtained by decomposing the pre-trained model can be adjusted accordingly to adapt it to the scoring rules in the specific scenario, thereby providing scenario adaptability for automatic scoring.

[0113] Next, based on this score point analysis, the second user's answer can be scored to obtain a predicted score and a score analysis. That is, based on the score point analysis, a pre-trained model can be applied to score the second user's answer, analyze the second user's answer to determine their answer performance, such as the score performance on each score point, the errors, the reasons for the errors, and the remedial measures, and determine the final score, i.e., the predicted score, while providing a score analysis.

[0114] In this embodiment of the invention, fine-tuning is performed on the scoring task based on the pre-trained model, enabling the model to answer the input second sample test question in a natural language dialogue manner. It can also score the second user's answer according to the reference answer and the given scoring criteria, pointing out where the second user's answer has errors and the reasons for the errors, and finally giving a predicted score and score analysis to explain the reason why the model gives this predicted score, thereby improving the credibility of the score and facilitating subsequent targeted learning.

[0115] Based on the above embodiments, step 120 includes:

[0116] Based on the pre-trained data, the initial model is applied to predict character units one by one to obtain the prediction results corresponding to the pre-trained data.

[0117] Based on the prediction results and pre-training data, the initial model is trained to obtain a pre-trained model, which is built on the basis of a large language model.

[0118] Specifically, step 120, the process of training the initial model based on the pre-training data to obtain the pre-trained model, may include:

[0119] After obtaining the pre-training data, before training the model, an initial model needs to be built in advance. Considering the problem of low accuracy in traditional scoring schemes, in this embodiment of the invention, when building the initial model, a large language model with good understanding of natural language and containing more knowledge can be used. Based on this large language model, the input text is accurately understood, thereby distinguishing user answers with the same semantics but different grammars and the same grammar but different semantics, and thus having the ability to score accurately.

[0120] Based on this, in this embodiment of the invention, an initial model can be built on a large language model, and then the initial model can be pre-trained based on pre-training data to improve its understanding ability, thereby obtaining a pre-trained model. Specifically, based on the pre-training data, the initial model is used for prediction. Based on the text continuation ability of the initial model, the subsequent text is automatically predicted according to the preceding text, so that the corresponding text content can be predicted and output for the pre-training data, i.e., the prediction result. Then, based on the prediction result and the pre-training data, the initial model can be trained to obtain the trained model, i.e., the pre-trained model.

[0121] Here, when the initial model makes predictions, it can predict the content of the masked segments based on the input text. Then, based on the masked segments output by the model and the actual masked segments, the model's loss on the prediction task can be evaluated, and the model can be trained accordingly to obtain the trained model. The masked segments can be individual or all. When all segments are masked, the prediction process involves predicting the token for the next time step based on the preceding text.

[0122] However, in traditional solutions, when predicting the token at the next moment, although the expression scenarios and semantics of the corpus can be learned from the massive corpus, in the scoring scenario, the numbers often have clear numerical meanings. Therefore, learning at the token (the smallest unit of corpus segmentation) level will lead to an inability to understand the numerical meaning expressed in the corpus.

[0123] In this embodiment of the invention, in order to enable the model to not only accurately predict the token (the smallest unit of corpus segmentation) at the next moment, but also to understand the specific meaning represented by the numbers in the pre-training data in the scoring scenario, when performing the prediction task, not only predicting the token at the next moment, but also predicting the numbers composed of the tokens, the difference between the prediction and the real numbers can be confirmed, and finally the prediction result corresponding to the pre-training data can be obtained.

[0124] Specifically, this can be done by applying an initial model to predict tokens character by character based on pre-trained data. Furthermore, when the next set of tokens forms a number, the difference between the predicted number and the actual number is also predicted to confirm whether the model's understanding of the number is correct and whether it has accurately learned the specific meaning represented by the number. This can greatly help improve the model's scoring accuracy in subsequent scoring tasks.

[0125] Next, the initial model can be trained based on the prediction results output by the model to obtain a trained initial model. Specifically, the initial model is self-supervised trained using the model's prediction output to obtain a pre-trained model. Here, the difference between the predicted results and the pre-training data is determined, and the model's loss is determined based on this difference. Parameters are then adjusted based on the loss to ensure that the model's prediction results are as close as possible to, or even identical to, the pre-training data, thus obtaining a trained pre-trained model.

[0126] Based on the above embodiments, the initial model is trained based on the prediction results and pre-training data to obtain a pre-trained model, including:

[0127] Based on the prediction results and pre-training data, determine the prediction loss;

[0128] Based on each predicted number in the prediction results and each number in the pre-training data, determine the numerical loss;

[0129] The initial model is trained based on prediction loss and numerical loss to obtain a pre-trained model.

[0130] Specifically, the process of training the initial model based on the prediction results and pre-training data to obtain the pre-trained model can include:

[0131] Since in the prediction task, the initial model not only predicts the token at the next time step, but also predicts the difference between the number and the actual number when the next group of tokens forms a number, in this embodiment of the invention, when measuring the loss of the model on the prediction task based on the model's prediction results and pre-training data, it can be considered from two aspects: the overall prediction results and the accuracy of the number prediction when the tokens form a number. Based on the loss on these two aspects, the overall loss of the model on the prediction task can be determined.

[0132] Specifically, firstly, based on the prediction results and pre-training data, the difference between the predicted tokens of the model at each time step and their corresponding true values ​​can be determined. Based on this difference, the prediction loss of the model can be determined. At the same time, based on each predicted number in the model's prediction results and each number in the pre-training data, the accuracy of the model's prediction of the numbers can be determined, that is, whether each number is predicted correctly, thus obtaining the number loss.

[0133] Furthermore, based on this, the predicted loss and numerical loss can be determined to establish the overall loss of the model in the prediction task. Then, based on this overall loss, the parameters of the initial model can be iterated to adjust the model parameters so that the prediction results output by the model after parameter adjustment can be as close as possible to the pre-training data, while ensuring that the numbers predicted by the model are consistent with the real numbers. Finally, the pre-trained model can be trained.

[0134] Figure 3 This is an example diagram of the pre-training process provided by the present invention, such as... Figure 3 As shown, token(t1,t2) form a number, denoted as num0, tokent4 is also a number, denoted as num1, and so on, to obtain K numbers in the pre-training data.

[0135] The goal of pre-training is to minimize the following loss function:

[0136]

[0137] In the formula, L represents the overall loss of the model on the prediction task, and the first term is used to calculate the loss given t0, t1, ..., t. i-1 The next token iThe probability of making a correct prediction, the second term is the predicted number num′. i With the real number num i The gap.

[0138] This invention also provides a scoring method. Figure 4 This is a flowchart illustrating the scoring method provided by the present invention, as shown below. Figure 4 As shown, the method includes:

[0139] Step 410: Determine the user's answer to be scored, and the corresponding question for the user's answer;

[0140] Step 420: Based on the test questions, apply the scoring model to score the user's answers to obtain the answer scores and score analysis; the score analysis is used to explain the composition of the answer scores; the scoring model is determined based on the training method of the scoring model as described in any of the above items.

[0141] Specifically, before scoring, it is first necessary to determine the user answers that need to be automatically scored, i.e., the user answers to be scored. This can specifically involve obtaining the user answers to be scored and the corresponding test questions. The test questions can be obtained by acquiring images of existing paper test questions through an image acquisition device, and then extracting text from the test question images. Alternatively, they can be obtained through web search or crawling, or directly downloaded / screenshotted from a specific test question website, or found from a test question bank. This embodiment of the invention does not specifically limit these methods.

[0142] Correspondingly, the user's answer to be scored can be obtained by capturing the image of the user's answer to the question through an image acquisition device and extracting the answer, or it can be obtained together with the question. That is, the question and user's answer can be obtained directly from a question bank or a specific website that carries the user's answer, and used as the user's answer to be scored and the corresponding question. This embodiment of the invention does not make specific limitations in this regard.

[0143] Here, the user's answer to be scored can correspond to one or more questions. If there are multiple questions, the user's answer to be scored can be one or more. For example, for one question, there may be multiple solutions. If there are multiple user answers and / or their corresponding questions, each user answer under each question can be scored separately to obtain the answer score and score analysis for each user answer.

[0144] Furthermore, after determining the user's answer to be rated, it can be rated to obtain the corresponding answer score and score analysis. That is, a pre-trained rating model can be applied to rate the user's answer to obtain the answer score and score analysis of the user's answer.

[0145] Specifically, this involves inputting the user's answer and its corresponding question into the scoring model. The model first understands and answers the input question, outputting a predicted answer as a reference. Then, based on learned answer analysis methods, it breaks down the reference answer into scoring points, obtaining a scoring point analysis. This analysis is then used to analyze the user's answer, determining its score, errors, reasons for errors, and remedial measures. Ultimately, the model outputs the answer score and scoring analysis. Here, the scoring analysis explains the composition of the answer score, providing a basis for the model's answer score.

[0146] Here, the scoring analysis can include the user's answer score, such as which scoring points were scored, the differences in scores between different parts, errors, such as which scoring points were missed, the reasons for the errors, and remedial measures.

[0147] After obtaining the answer scores and analysis from users' answers, further intervention in learning can be conducted using these scores and analysis. For example, based on the user's answer scores, score distribution, and reasons for errors, targeted learning materials can be constructed. Specifically, when a calculation error occurs, calculation-focused reinforcement exercises related to the same knowledge point as the original question can be created and recommended to the user; when an answer is not in accordance with the correct format, corresponding explanatory texts, videos, and images can be created and pushed to the user. Then, while providing feedback on the answer score, suggestions for reinforcement plans can be offered to the user. If the user accepts the suggestions, corresponding learning materials can be pushed to them to facilitate their learning consolidation.

[0148] The scoring model can be trained using the following steps: First, pre-training data and training data are acquired. The pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score for the first sample test question. The training data includes scoring specifications, a second sample test question, and its corresponding second user answer. Based on the pre-training data, the initial model is trained to obtain a pre-trained model. Based on the second sample test question and scoring specifications, the pre-trained model is applied to score the second user answer, obtaining a predicted score and a score analysis for the second user answer. The score analysis explains the composition of the predicted score. Finally, based on the training data, the predicted score, and the score analysis, the pre-trained model is trained to obtain the scoring model.

[0149] The scoring model trained using the aforementioned methods possesses strong natural language processing capabilities, enabling accurate understanding of natural language. Therefore, it addresses the issue of low scoring accuracy in traditional methods due to grammatical and syntax limitations. Furthermore, for situations with multiple solutions to a single question, this embodiment of the invention employs a human-like understanding ability, possessing problem-solving skills. When a user's answer differs from the reference answer, the model can, to some extent, independently determine the correctness of the user's answer, rather than relying entirely on the standard answer. Moreover, the explanatory scoring analysis output by the model not only supports the predicted answer score but also identifies errors and their reasons, making human-computer interaction more natural and fluid.

[0150] The scoring method provided by this invention uses a scoring model trained through a scoring model training method to score user answers, obtaining answer scores and score analysis. This overcomes the shortcomings of traditional schemes, such as low scoring accuracy and the fact that only an overall score is obtained after scoring, which is not conducive to further learning. It realizes automatic scoring of user answers and provides score analysis. While ensuring scoring accuracy and efficiency, the score analysis can help understand the user's answer performance, thereby providing data support for subsequent consolidation learning.

[0151] Based on the above embodiments, the scoring analysis includes the error score points corresponding to the answer score and the reasons for the errors; step 420 includes:

[0152] Based on the test questions, user answers, and error types, generate scoring prompts.

[0153] Input the scoring prompt statement into the scoring model to obtain the answer score output by the scoring model, as well as the error type corresponding to the error point and the reason for the error.

[0154] Specifically, step 420, which involves applying a scoring model to score the user's answer based on the test questions, and obtaining the answer score and score analysis, may include:

[0155] When the model scores user answers, it typically provides an analytical description or explanatory statement regarding the reasons for errors, rather than directly identifying specific error types such as typos, calculation errors, or non-standard answers. This makes further tasks such as error summarization and statistical analysis very inconvenient and requires human understanding and processing.

[0156] In this embodiment of the invention, when applying the scoring model for scoring, the error reasons in the scoring analysis output by the model can be limited by type, so that the analyzed error reasons are directly mapped to specific error types and output accordingly, so as to facilitate the next step of analysis and statistics.

[0157] Specifically, when scoring user answers based on a scoring model, prompt statements that the model can receive and understand can be generated first based on the question and the user's answer. In order for the model to directly correspond the error reason to the specific error type when outputting the score analysis, in this embodiment of the invention, the error type can be taken into consideration when generating the prompt statement. That is, a scoring prompt statement, such as a prompt, can be generated based on the question, the user's answer, and the error type.

[0158] Here, the scoring prompts can contain keywords corresponding to each type of error. For example, typos can correspond to extra words, missing words, misspellings, etc.; calculation errors, etc.; non-standard answers can correspond to reversed cause and effect, incorrect step order, etc.; and calculation errors can correspond to incorrect calculation ideas, incorrect calculation process, incorrect calculation results, etc.

[0159] Then, this scoring prompt can be input into the scoring model so that the scoring model can understand and respond to the content of the input scoring prompt. The model scores the user's answer based on the predicted reference answer. That is, by breaking down the predicted reference answer into scoring points, analyzing the user's answer based on the analysis of the broken scoring points, determining the answer score, and the scoring analysis. The scoring analysis here includes the user's error situation, i.e., the error scoring points, and the error type of the error reason corresponding to the error scoring points.

[0160] The training apparatus for the scoring model provided by the present invention will be described below. The training apparatus for the scoring model described below can be referred to in correspondence with the training method for the scoring model described above.

[0161] Figure 5 This is a schematic diagram of the training device for the scoring model provided by the present invention, as shown below. Figure 5 As shown, the device includes:

[0162] The acquisition unit 510 is used to acquire pre-training data and training data. The pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score of the first sample test question. The training data includes a scoring specification, a second sample test question, and its corresponding second user answer.

[0163] The pre-training unit 520 is used to train the initial model based on the pre-training data to obtain a pre-trained model;

[0164] The prediction unit 530 is used to score the second user's answer based on the second sample test questions and the scoring criteria, using the pre-trained model to obtain the predicted score and score analysis of the second user's answer, wherein the score analysis is used to explain the composition of the predicted score.

[0165] Training unit 540 is used to train the pre-trained model based on the training data, the predicted score, and the score analysis to obtain a scoring model.

[0166] The training device for the scoring model provided by this invention improves the model's understanding of input text through pre-training, enabling accurate understanding of the semantics of the input text and thus accurate answering and scoring. Based on this, fine-tuning of the scoring is performed to better adapt the model to the question-answering task. It can output predicted scores and score analyses based on the input questions, scoring criteria, and user answers, explaining the composition of the predicted scores and providing a basis for their delivery. This overcomes the shortcomings of traditional automatic scoring schemes, such as the difficulty in comprehensively and accurately understanding text semantics, leading to low scoring accuracy, and the fact that only an overall score is obtained, which is not conducive to further learning. This invention achieves automatic scoring of user answers and provides score analyses, ensuring scoring accuracy and efficiency while providing data support for subsequent consolidation learning.

[0167] Based on the above embodiments, the prediction unit 530 is used for:

[0168] Based on the second sample test questions, the pre-trained model is used to generate answers, and the predicted answers corresponding to the second sample test questions are obtained.

[0169] Based on the predicted answer and the scoring criteria, the pre-trained model is used to score the second user's answer, resulting in a predicted score and a score analysis.

[0170] Based on the above embodiments, the prediction unit 530 is used for:

[0171] Based on the scoring criteria, the pre-trained model is used to decompose the predicted answer into scoring points to obtain the scoring point analysis;

[0172] Based on the score point analysis, the pre-trained model is applied to score the second user's answer, resulting in a predicted score and a score analysis.

[0173] Based on the above embodiments, the pre-training unit 520 is used for:

[0174] Based on the pre-trained data, the initial model is applied to predict character units one by one to obtain the prediction results corresponding to the pre-trained data.

[0175] Based on the prediction results and the pre-training data, the initial model is trained to obtain a pre-trained model, which is built on the basis of a large language model.

[0176] Based on the above embodiments, the pre-training unit 520 is used for:

[0177] Based on the prediction results and the pre-training data, determine the prediction loss;

[0178] Based on each predicted number in the prediction results and each number in the pre-training data, determine the digital loss;

[0179] Based on the prediction loss and the digital loss, the initial model is trained to obtain a pre-trained model.

[0180] The scoring device provided by the present invention will be described below. The scoring device described below can be referred to in correspondence with the scoring method described above.

[0181] Figure 6 This is a schematic diagram of the scoring device provided by the present invention, as shown below. Figure 6 As shown, the device includes:

[0182] The determining unit 610 is used to determine the user's answer to be scored, and the test question corresponding to the user's answer;

[0183] Scoring unit 620 is used to score the user's answer based on the test question using a scoring model, and obtain an answer score and a score analysis; the score analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any of the above.

[0184] The scoring device provided by this invention uses a scoring model trained through a scoring model training method to score user answers, obtaining answer scores and score analysis. This overcomes the shortcomings of traditional schemes, such as low scoring accuracy and the fact that only an overall score is obtained after scoring, which is not convenient for further learning. It realizes automatic scoring of user answers and provides score analysis. While ensuring scoring accuracy and efficiency, the score analysis can help understand the user's answer performance, thereby providing data support for subsequent consolidation learning.

[0185] Based on the above embodiments, the scoring analysis includes the error score points corresponding to the answer score and the reasons for the errors; the scoring unit 620 is used for:

[0186] Based on the test questions, the user's answers, and the error types, a scoring prompt statement is generated;

[0187] The scoring prompt statement is input into the scoring model to obtain the answer score output by the scoring model, as well as the error type corresponding to the error point and the error reason.

[0188] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a training method or a scoring method for a scoring model. The training method for the scoring model includes: acquiring pre-training data and training data; the pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score for the first sample test question; the training data includes a scoring specification, a second sample test question, and its corresponding second user answer; training an initial model based on the pre-training data to obtain a pre-trained model; applying the pre-trained model to score the second user answer based on the second sample test question and the scoring specification to obtain a predicted score and a score analysis for the second user answer, wherein the score analysis is used to explain the composition of the predicted score; and training the pre-trained model based on the training data, the predicted score, and the score analysis to obtain a scoring model. The scoring method includes: determining the user's answer to be scored, and the test question corresponding to the user's answer; applying a scoring model to score the user's answer based on the test question, to obtain an answer score and a scoring analysis; the scoring analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any of the above.

[0189] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the training method or scoring method of the scoring model provided by the above methods, wherein the training method of the scoring model comprises: acquiring pre-training data and training data, the pre-training data comprising a first sample test question, and a reference answer, a first user answer, and an answer score corresponding to the first sample test question; the training data comprising a scoring specification, a second sample test question, and a corresponding second user answer; training an initial model based on the pre-training data to obtain a pre-trained model; applying the pre-trained model to score the second user answer based on the second sample test question and the scoring specification to obtain a predicted score and a score analysis for the second user answer, the score analysis being used to explain the composition of the predicted score; and training the pre-trained model based on the training data, the predicted score, and the score analysis to obtain a scoring model. The scoring method includes: determining the user's answer to be scored, and the test question corresponding to the user's answer; applying a scoring model to score the user's answer based on the test question, to obtain an answer score and a scoring analysis; the scoring analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any of the above.

[0191] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a training method or a scoring method for the scoring model provided by the methods described above. The training method for the scoring model includes: acquiring pre-training data and training data, wherein the pre-training data includes a first sample test question, and a reference answer, a first user answer, and an answer score corresponding to the first sample test question; the training data includes a scoring specification, a second sample test question, and a corresponding second user answer; training an initial model based on the pre-training data to obtain a pre-trained model; applying the pre-trained model to score the second user answer based on the second sample test question and the scoring specification to obtain a predicted score and a score analysis for the second user answer, wherein the score analysis is used to explain the composition of the predicted score; and training the pre-trained model based on the training data, the predicted score, and the score analysis to obtain a scoring model. The scoring method includes: determining the user's answer to be scored, and the test question corresponding to the user's answer; applying a scoring model to score the user's answer based on the test question, to obtain an answer score and a scoring analysis; the scoring analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any of the above.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a scoring model, characterized in that, include: Acquire pre-training data and training data. The pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score for the first sample test question. The training data includes scoring specifications, a second sample test question, and its corresponding second user answer. Based on the pre-training data, the initial model is trained to obtain the pre-trained model; Based on the second sample test questions and the scoring criteria, the pre-trained model is applied to score the second user's answer to obtain the predicted score and score analysis of the second user's answer. The score analysis is used to explain the composition of the predicted score. Based on the training data, the predicted score, and the score analysis, the pre-trained model is trained to obtain the scoring model; The step of training the initial model based on the pre-trained data to obtain the pre-trained model includes: Based on the pre-trained data, the initial model is applied to predict character units one by one to obtain the prediction results corresponding to the pre-trained data. Based on the prediction results and the pre-training data, the initial model is trained to obtain a pre-trained model, which is built on the basis of a large language model.

2. The method of Claim 1, wherein, The step of scoring the second user's answer based on the second sample test questions and the scoring criteria, using the pre-trained model to score the second user's answer, and obtaining the predicted score and score analysis of the second user's answer, includes: Based on the second sample test questions, the pre-trained model is used to generate answers, and the predicted answers corresponding to the second sample test questions are obtained. Based on the predicted answer and the scoring criteria, the pre-trained model is used to score the second user's answer, resulting in a predicted score and a score analysis.

3. The method of Claim 2, wherein, The step of scoring the second user's answer using the pre-trained model based on the predicted answer and the scoring criteria to obtain the predicted score and score analysis includes: Based on the scoring criteria, the pre-trained model is used to decompose the predicted answer into scoring points to obtain the scoring point analysis; Based on the score point analysis, the pre-trained model is applied to score the second user's answer, resulting in a predicted score and a score analysis. 4.The method of Claim 1, wherein, The step of training the initial model based on the prediction results and the pre-training data to obtain a pre-trained model includes: Based on the prediction results and the pre-training data, determine the prediction loss; Based on each predicted number in the prediction results and each number in the pre-training data, determine the digital loss; Based on the prediction loss and the digital loss, the initial model is trained to obtain a pre-trained model.

5. A method of scoring, characterized by, include: Determine the user's answer to be scored, and the corresponding question for that answer; Based on the test questions, a scoring model is applied to score the user's answers, resulting in answer scores and score analysis; the score analysis is used to explain the composition of the answer scores. The scoring model is determined based on the training method of the scoring model as described in any one of claims 1 to 4.

6. The scoring method of claim 5, wherein, The scoring analysis includes the error points corresponding to the answer score and the reasons for the errors; The process of applying a scoring model to score the user's answer based on the test question, resulting in an answer score and score analysis, includes: Based on the test questions, the user's answers, and the error types, a scoring prompt statement is generated; The scoring prompt statement is input into the scoring model to obtain the answer score output by the scoring model, as well as the error type corresponding to the error point and the error reason. 7.A device for training a scoring model, characterized in that, include: The acquisition unit is used to acquire pre-training data and training data. The pre-training data includes a first sample test question, and the corresponding reference answer, first user answer, and answer score for the first sample test question. The training data includes a scoring specification, a second sample test question, and its corresponding second user answer. A pre-training unit is used to train an initial model based on the pre-training data to obtain a pre-trained model; The prediction unit is used to score the second user's answer based on the second sample test questions and the scoring criteria, using the pre-trained model to obtain the predicted score and score analysis of the second user's answer, wherein the score analysis is used to explain the composition of the predicted score. A training unit is used to train the pre-trained model based on the training data, the predicted score, and the score analysis to obtain a scoring model; The step of training the initial model based on the pre-trained data to obtain the pre-trained model includes: Based on the pre-trained data, the initial model is applied to predict character units one by one to obtain the prediction results corresponding to the pre-trained data. Based on the prediction results and the pre-training data, the initial model is trained to obtain a pre-trained model, which is built on the basis of a large language model.

8. A scoring device, characterized in that, include: A determining unit is used to determine the user's answer to be scored, and the test question corresponding to the user's answer; A scoring unit is used to score the user's answer based on the test question using a scoring model, thereby obtaining an answer score and a score analysis; the score analysis is used to explain the composition of the answer score; the scoring model is determined based on the training method of the scoring model as described in any one of claims 1 to 4.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the training method of the scoring model as described in any one of claims 1 to 4, or the scoring method as described in claim 5 or 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the scoring model as described in any one of claims 1 to 4, or the scoring method as described in claim 5 or 6.

Images