Method for enhancing teaching evaluation credibility in large education model

Abstract

The invention discloses a method for enhancing teaching evaluation credibility in an education large model, and the method comprises the steps: obtaining a scoring prompt, and obtaining first scoring evaluation information of answer content according to the scoring prompt; obtaining an interaction object corresponding to the first score evaluation information, and constructing scores of different users into a historical score sequence under the same score standard; obtaining a dynamic smoothing coefficient according to the historical score sequence; obtaining the feedback quality of the first score evaluation information according to the historical score sequence and the dynamic smoothing coefficient; and under the condition that the feedback quality of the first score evaluation information is high, determining that the first score evaluation information reaches the standard, and sending the first score evaluation information to the first terminal. And quantitatively evaluating the feedback quality of the first score evaluation information through the change of the dynamic proportion of the new and old information. Therefore, the fluctuation of scoring results of different batches and different times is controllable, and a scoring basis with high interpretability is formed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent teaching technology, and in particular to a method for enhancing the credibility of teaching evaluation in a large education model. Background Art

[0002] Currently, in educational big models, technologies that automatically score answers based on large language models (such as GPT and BERT) primarily rely on the semantic judgment of a single answer based on the large language model. This fails to fully consider changes in historical scores or model scoring confidence over time. This leads to significant fluctuations in scoring results between different time periods or batches, impacting the stability and reliability of educational assessments. The approach of "feeding student answers into a large language model, performing semantic understanding of the answers, and finally generating a scoring result" results in poorly interpretable scoring criteria, making it difficult for teachers and students to effectively understand the scoring results. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for enhancing the credibility of teaching evaluation in an educational macro-model, thereby achieving the effect of small fluctuations in scoring results and good interpretability of the scoring basis.

[0004] According to one aspect of the present invention, a method for enhancing the credibility of teaching evaluation in an educational macro model is provided, the method comprising: Obtaining a scoring prompt, and obtaining first scoring evaluation information of the answer content according to the scoring prompt; Obtaining the interactive object corresponding to the first rating evaluation information, and constructing the scores of different users into a historical score sequence under the same rating standard; Obtaining a dynamic smoothing coefficient based on the historical performance sequence; Obtaining a current matching degree value edge weight of the first scoring evaluation information according to the historical score sequence and the dynamic smoothing coefficient; Obtaining a confidence value of the first rating evaluation information according to the current matching degree numerical edge weight; Obtaining feedback quality of the first scoring evaluation information according to the confidence value; If the feedback quality of the first scoring evaluation information is high, it is determined that the first scoring evaluation information meets the requirements, and the first scoring evaluation information is sent to the first terminal.

[0005] Optionally, before obtaining the scoring prompt and obtaining first scoring evaluation information of the answer content according to the scoring prompt, the method further includes: Obtain teaching text data of the online teaching system; Preprocessing the acquired teaching text data to construct a teaching knowledge base, wherein the preprocessing includes text cleaning, text deduplication and text segmentation; Based on the teaching knowledge base, pre-training is performed on the large language model to obtain an initial state LLM large model; Get the question setting instructions; According to the question-setting instruction, the initial subjective question is generated by the initial state LLM large model.

[0006] Optionally, before obtaining the scoring prompt and obtaining first scoring evaluation information of the answer content according to the scoring prompt, the method further includes: Obtaining an initial subjective question and a rating scale corresponding to the answer content; wherein the answer content is the answers of multiple users to the initial subjective question; The step of obtaining a scoring prompt and obtaining first scoring evaluation information of the answer content according to the scoring prompt includes: Obtaining the scoring prompt and the scoring scale; The scoring prompt, the scoring scale, and the answer content are input into the initial LLM macro model to obtain the first scoring evaluation information of the answer content.

[0007] Optionally, the method further includes: When the feedback quality of the first scoring evaluation information is low, obtaining a high-quality data set; Based on the high-quality data set, the initial LLM large model is fine-tuned to obtain a fine-tuned LLM large model.

[0008] Optionally, fine-tuning the initial LLM large model based on the high-quality data set to obtain a fine-tuned LLM large model includes: Screening out text paragraphs containing descriptions of the causal relationship between the scoring criteria and the quality of the answer from the teaching knowledge base; Constructing a causal relationship dataset based on the scoring criteria and the causal relationship description of the answer quality in the text paragraph; Using the causal relationship dataset to reason and analyze the input subjective questions, and construct question-answer pairs; Based on the question-answer pair, the initial LLM large model is fine-tuned to obtain the fine-tuned LLM large model.

[0009] Optionally, the method further includes: Obtaining second scoring evaluation information of the answer content based on the fine-tuned LLM model; Sending the second rating evaluation information to the first terminal; or, Obtaining third scoring evaluation information of the answer content based on the fine-tuned LLM model; If it is determined that the feedback quality of the third scoring evaluation information is high, the third scoring evaluation information is sent to the first terminal.

[0010] Optionally, obtaining a dynamic smoothing coefficient based on the historical performance sequence includes: Calculate the standard deviation of the historical performance series; The dynamic smoothing coefficient is calculated according to a dynamic coefficient algorithm, wherein the dynamic coefficient algorithm is obtained by the following method:

[0011] in, represents the dynamic smoothing coefficient; represents the minimum smoothing factor; represents the maximum smoothing factor; Represents the standard deviation of the historical score series; base represents the upper threshold of the score.

[0012] Optionally, obtaining the current matching degree numerical edge weight of the first rating evaluation information according to the historical score sequence and the dynamic smoothing coefficient includes: Calculate the residual initialization weight according to the residual weight algorithm; Calculate the current score contribution value according to the score contribution algorithm; The current matching degree value edge weight is calculated according to the matching strength algorithm, wherein the matching strength algorithm is obtained by the following method:

[0013] in, Indicates the current matching value edge weight; represents the sovereign control coefficient; Indicates the current score contribution value; represents the residual initialization weight; Represents the initial matching value edge weight.

[0014] Optionally, obtaining a confidence value of the first rating evaluation information according to the edge weight of the current matching degree value; and obtaining feedback quality of the first rating evaluation information according to the confidence value include: The confidence value of the first scoring evaluation information is calculated according to a confidence algorithm, wherein the confidence algorithm is obtained by the following method:

[0015] Where Z represents the confidence value; a first score representing the first scoring evaluation information; represents the expected score; Determining whether the confidence value falls within a confidence interval; If the confidence value falls within the confidence interval, it is determined that the feedback quality of the first scoring evaluation information is high.

[0016] Optionally, after pre-training the large language model based on the teaching knowledge base to obtain the initial state LLM large model, the method further includes: Constructing a multidimensional feature vector set based on the teaching knowledge base, wherein the multidimensional feature vector set includes a dimension coverage vector, a standard description vector, and a domain adaptation vector; Setting the output structure rationality scoring standard according to the dimension coverage vector; setting the operability scoring standard according to the standard description vector; setting the domain adaptability scoring standard according to the domain adaptation vector; The question scoring value R is calculated according to the question scoring algorithm, wherein the question scoring algorithm is obtained by the following method: R= *Structural rationality score+ *Operability score+ *Domain suitability score; in, + + =1, and 、 、 Dynamically adjust according to application scenarios, 、 、 Both represent rating coefficients.

[0017] Optionally, the method further includes: Preprocessing the initial subjective questions, the answer content, and the first scoring evaluation information with high feedback quality to construct a second teaching knowledge base, wherein the preprocessing includes text cleaning, text deduplication, and text segmentation; Based on the second teaching knowledge base, pre-training is performed on the initial state LLM large model to obtain an advanced version of the LLM large model.

[0018] According to another aspect of the present invention, there is also provided a system for enhancing the credibility of teaching evaluation in an educational macro model, the system comprising an acquisition unit, a quality evaluation unit and a quality feedback unit; The obtaining unit is configured to obtain a scoring prompt, and obtain first scoring evaluation information of the answer content according to the scoring prompt; The quality evaluation unit is configured to obtain an interactive object corresponding to the first rating evaluation information, and construct the scores of different users into a historical score sequence under the same rating standard; obtain a dynamic smoothing coefficient based on the historical score sequence; obtain a current matching numerical edge weight of the first rating evaluation information based on the historical score sequence and the dynamic smoothing coefficient; obtain a confidence value of the first rating evaluation information based on the current matching numerical edge weight; and obtain feedback quality of the first rating evaluation information based on the confidence value; The quality feedback unit is configured to determine that the first scoring and evaluation information meets the requirements when the feedback quality of the first scoring and evaluation information is high, and send the first scoring and evaluation information to the first terminal.

[0019] Optionally, the system further includes a pre-training unit and a question setting unit; The pre-training unit is configured to obtain teaching text data from an online teaching system; pre-process the obtained teaching text data to construct a teaching knowledge base, wherein the pre-processing includes text cleaning, text deduplication, and text segmentation; and pre-train a large language model based on the teaching knowledge base to obtain an initial state LLM large model. The question-setting unit is used to obtain a question-setting instruction; according to the question-setting instruction, the initial state LLM large model generates an initial subjective question.

[0020] According to another aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned method for enhancing the credibility of teaching evaluation in an educational macro-model are implemented.

[0021] The present invention provides a method for enhancing the credibility of teaching evaluations in a large-scale educational model. The method involves obtaining a scoring prompt, obtaining first scoring and evaluation information about the answer content based on the scoring prompt, calculating a current matching numerical edge weight for the first scoring and evaluation information, and then calculating a confidence value for the first scoring and evaluation information based on the current matching numerical edge weight. Using the confidence value as a criterion, only first scoring and evaluation information with high feedback quality is output. In this technical solution, based on the first scoring and evaluation information, the scores of different users under the same scoring criteria are constructed into a historical score sequence. A dynamic smoothing coefficient is calculated based on the historical score sequence. Based on the historical score sequence and the dynamic smoothing coefficient, the current matching numerical edge weight for the first scoring and evaluation information is obtained. The current matching numerical edge weight is composed of the current scoring contribution value (new information) and the residual initialization weight (old information). The feedback quality of the first scoring and evaluation information is quantitatively assessed by dynamically changing the ratio of new and old information in real time. By converting abstract scoring and evaluation information into data representation, objective data is used to reflect the true feedback quality. Dynamic adjustment of the ratio of new and old information ensures that fluctuations in scoring results across different batches and over time are manageable, forming a highly interpretable scoring basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be construed as limiting the present invention. The same reference numerals are used throughout the accompanying drawings to denote the same components.

[0023] Figure 1 A flow chart of a method for enhancing the credibility of teaching evaluation in an educational macro model provided by the present invention.

[0024] Figure 2 A schematic diagram of the association mapping relationship of the teaching text data provided by the present invention.

[0025] Figure 3 This is a schematic diagram of the scoring table structure constructed based on the answer content provided by the present invention.

[0026] Figure 4 Another flow chart of a method for enhancing the credibility of teaching evaluation in an educational macro model provided by the present invention.

[0027] Figure 5 This is a schematic diagram of displaying subjective questions on a terminal provided by the present invention.

[0028] Figure 6 A schematic diagram of the composition structure of a system for enhancing the credibility of teaching evaluation in an educational model provided by the present invention.

[0029] Figure 7A schematic diagram of the hardware architecture of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0032] In a first aspect, an embodiment of the present invention provides a method for enhancing the credibility of teaching evaluation in an educational macro model, which is applied to a client (e.g., a first terminal). The method is executed by the client, and the processing flow of the client specifically includes the following steps: Step S01: Obtain initial subjective questions.

[0033] In this embodiment, the initial subjective questions are sent to a client, for example, a first terminal. The first terminal can be a computer device used by a teacher or a computer device used by a student. Specifically, the initial subjective questions are generated by a pre-trained initial state LLM large model. Subjective questions refer to the type of questions that require candidates to think independently, organize language, express opinions, or express themselves creatively during the answering process. Its core feature is that the answers are not unique and there is usually no fixed standard answer. It focuses more on testing the comprehensive qualities of the candidates, such as their thinking process, analytical ability, knowledge application ability, and language expression ability.

[0034] LLM stands for Large Language Model in English and is also known as "Large Language Model" in Chinese. For example, the large language model can be any of chatGPT, Wenxin Yiyan, Tongyi Qianwen, or Deepseek.

[0035] Step S02: Obtain a rating scale corresponding to the answer content, wherein the initial subjective question is the subjective question corresponding to the answer content.

[0036] In this embodiment, the rating scale is a rating rubric, and its full English name is Rubric. The rating scale is a structured rating tool used to systematically evaluate subjective questions. The rating scale provides an objective and operational framework for the rating process by clearly defining the rating criteria and rating levels (including scores), thereby reducing the subjectivity of rating and improving the consistency and fairness of the evaluation. In an embodiment of the present invention, the initial state LLM large model can automatically generate a rating scale based on the rating logic in the pre-trained teaching knowledge base. Exemplarily, the rating scale can be in JSON / XML format.

[0037] It should be noted that there is a one-to-one correspondence between rating scales and subjective questions. That is, each subjective question corresponds to a rating scale. For example, subjective question X corresponds to rating scale x, and subjective question Y corresponds to rating scale y. Rating scales x and y are different.

[0038] Step S03: Obtain the answer content for the initial subjective question.

[0039] In this embodiment, the initial subjective question is sent to the computer device used by the student. After viewing the initial subjective question on the computer device, the student enters the answer content (ie, the answer content) to the initial subjective question into the computer device.

[0040] like Figure 1 As shown: After step 03, the client's processing flow specifically includes the following steps: Step S101: Obtain a scoring prompt, and obtain first scoring evaluation information of the answer content according to the scoring prompt.

[0041] In this embodiment, this step includes obtaining a scoring prompt and a scoring scale, inputting the scoring prompt, scoring scale, and answer content into the initial LLM macromodel, and obtaining first scoring and evaluation information for the answer content. The scoring prompt triggers a scoring operation based on a prompt (full name in English). The scoring prompt represents a prompt input used to guide scoring or generate a score. For example, the scoring prompt can be a prompt inputted by the teacher into the computer device: "Generate scoring and evaluation information based on the answer content inputted by the student." Alternatively, the scoring prompt can be generated by the initial LLM macromodel.

[0042] Prompt means "a prompt" or "an input instruction." In the field of artificial intelligence, especially natural language processing (NLP) and chatbots, a prompt is text input by the user that guides the AI ​​to generate an answer or perform a task. For example, when user A asks a question to a system, the question is the prompt, and the system generates an answer based on the prompt.

[0043] After receiving the scoring prompt, the initial state LLM model will generate first scoring evaluation information for the student's answer. Specifically, the first scoring evaluation information includes a first score and a first evaluation feedback. That is, the first score is the score assigned by the initial state LLM model.

[0044] Step S102: obtaining the interactive object corresponding to the first scoring evaluation information, and constructing the scores of different users into a historical score sequence under the same scoring standard.

[0045] In this embodiment, the interactive objects refer to the initial subjective question (object 1), the answer content for the initial subjective question (object 2), the identity of the student who submitted the answer content (which can be recorded by student ID, object 3), and the scoring criteria (object 4). Among them, the scoring criteria represents the specific angle of scoring, such as Figure 3 As shown, the scoring criteria include content completeness, language expression, structural organization, and innovation. By obtaining the interaction object, the student's identity can be determined and a correspondence can be established between the student's identity and the content of their answer. This allows for subsequent scoring and evaluation to be accurately recorded for the corresponding student.

[0046] Based on the timestamp or scoring record identifier in the first scoring evaluation information, the current scoring moment is accurately obtained; with this moment as the dividing point, the scoring data of a preset number of times (for example, 5 times) under the same scoring standard before this moment is retroactively extracted. This scoring data comes from the scores of different users and is defined as historical scoring data. For example, if the interactive object includes the "language expression" scoring standard, then the scoring data of 5 times by different users that belong to the "language expression" scoring standard before this moment is retroactively extracted. Based on the historical scoring data, a historical score sequence is constructed. Different users can be multiple students. Students can be distinguished based on their student numbers or based on the product unique identifier (ID) of the terminal device they use; they can also be multiple faculty members, multiple employees, and so on.

[0047] Step S103: Obtain a dynamic smoothing coefficient based on the historical performance sequence.

[0048] In this embodiment, based on the fluctuation characteristics and trend analysis of the historical performance series, the adaptive module calculates a dynamic smoothing coefficient, which is used to quantify the stability of the historical performance.

[0049] Step S104: obtaining the current matching degree value edge weight of the first scoring evaluation information according to the historical score sequence and the dynamic smoothing coefficient.

[0050] Step S105: Obtain the confidence value of the first scoring evaluation information according to the current matching degree value edge weight.

[0051] Step S106: Obtaining feedback quality of the first scoring evaluation information according to the confidence value.

[0052] In this embodiment, based on the statistical characteristics (standard deviation) of the historical score sequence and a dynamic smoothing coefficient, an adaptive module calculates the current matching numerical edge weight, which includes the current score contribution value (new information) and the residual initialization weight (old information). The current score contribution value reflects the real-time contribution of the current score, while the residual initialization weight reflects the cumulative impact of historical scores. The proportion of the current score contribution value and the residual initialization weight in the current matching numerical edge weight changes dynamically. By dynamically changing the ratio of new and old information in real time, the feedback quality of the first scoring evaluation information is quantitatively evaluated. By converting abstract scoring evaluation information into data representation, objective data is used to reflect the actual feedback quality. By dynamically adjusting the ratio of new and old information, fluctuations in scoring evaluation information for each answer content, across different batches and at different times, under the same scoring criteria can be controlled, forming a highly interpretable scoring basis.

[0053] like Figure 3 As shown, the scoring criteria include four dimensions: content completeness, language expression, structural organization, and innovation. In this embodiment, all of the above steps are performed based on the same scoring criteria. The first scoring evaluation information refers to the score and feedback on the answer content under a specific scoring criterion (e.g., language expression). This technical solution requires generating corresponding scoring evaluation information for each of the four scoring criteria in sequence, and ultimately accumulating the scoring evaluation information for each of the four scoring criteria to form a comprehensive score.

[0054] Step S107: If the feedback quality of the first scoring evaluation information is high, determine that the first scoring evaluation information meets the requirements, and send the first scoring evaluation information to the first terminal.

[0055] In this embodiment, if the feedback quality of the first scoring and evaluation information is high (e.g., small fluctuations and high consistency with historical trends), the first scoring and evaluation information is sent to the first terminal, and the valid scoring and evaluation information is output. For example, the first scoring and evaluation information is displayed at a location corresponding to the answer content displayed on the first terminal, where the first scoring and evaluation information includes the first score and the first evaluation feedback. The location corresponding to the answer content displayed on the first terminal can be the next line below the answer content, the top of the answer content, and so on.

[0056] In this technical solution, through quantitative analysis of historical performance sequences, abstract scoring and evaluation information is converted into data expression, and the real feedback quality is reflected with objective data; through dynamic adjustment of the ratio of new and old information, the fluctuations of scoring results in different batches and at different times are ensured to be controllable, forming a highly interpretable scoring basis.

[0057] In one embodiment of the present invention, before obtaining the scoring prompt and obtaining the first scoring evaluation information of the answer content according to the scoring prompt, the method further includes the following steps: Step S201: Acquire teaching text data of the online teaching system.

[0058] In this embodiment, the data to be preprocessed may be at least one of text data, picture data, video data, audio data, formula data, and graphic data.

[0059] The introduction starts with preprocessing based on teaching text data. Specifically, the teaching text data of the online teaching system is obtained. The teaching text data includes the definitions of concepts such as courses, majors, subjects, course objectives, syllabuses, class hours, training objectives and graduation requirements, as well as teaching content, and the association mapping relationship between various teaching text data. The association mapping relationship includes concept subordination, content inclusion and logical derivation relationship.

[0060] like Figure 2 As shown, the online teaching system records the definitions of the concepts of courses, majors, disciplines, course objectives, syllabus, class hours, training objectives and graduation requirements, as well as the teaching content (corresponding to Figure 2 The association mapping relationship between the above items is recorded. The association mapping relationship includes concept subordination, content inclusion and logical derivation.

[0061] For example, courses are determined based on majors, so the association mapping relationship between courses and majors is a concept-subordinate relationship; majors are determined based on disciplines, so the association mapping relationship between majors and disciplines is a concept-subordinate relationship.

[0062] For example, since the syllabus covers the teaching content, the association mapping relationship between the syllabus and the content is a content inclusion relationship.

[0063] For example, graduation requirements are clarified based on training objectives, so the correlation mapping relationship between training objectives and graduation requirements is a logical derivative relationship.

[0064] Optionally, the initial state LLM large model includes a vector database unit, a knowledge base unit, and a crawling unit. The knowledge base unit is used to store the teaching text data of the online teaching system. The crawling unit is used to crawl the description content of the training objectives, graduation requirements, syllabus, class hours, subjective questions and answer content. The crawling unit uses technical means such as crawlers. The vector database unit converts the teaching text data into a high-dimensional vector (Embedding) through an encoder. Each dimension in the vector represents a semantic feature (such as keywords, emotional tendencies, contextual relationships, etc.). Based on measurement methods such as cosine similarity and Euclidean distance between vectors, it quickly finds the historical data or knowledge base content that is closest to the current input semantics.

[0065] Step S202: pre-processing the acquired teaching text data to construct a teaching knowledge base, wherein the pre-processing includes text cleaning, text deduplication and text segmentation.

[0066] In this embodiment, preprocessing refers to the entire process of systematically processing raw text data before large-scale model training, with the goal of converting unstructured text into standardized input that the model can understand. Text cleaning, text deduplication, and text segmentation are all core operations in preprocessing. The three respectively address data quality, redundancy, and structural standardization issues, and together constitute the basic framework of preprocessing. Text cleaning, text deduplication, and text segmentation all use processing methods well known in the art. Among them, the raw text data can be text comprehension from images, text / table extraction from images, speech recognition to text, and so on.

[0067] Step S203: Based on the teaching knowledge base, pre-training is performed on the large language model to obtain an initial state LLM large model.

[0068] Specifically, large-scale model pre-training refers to the process of initial training of deep learning models on large-scale general data, aiming to enable the model to automatically learn the common patterns, semantic representations, and structural regularities in the data. The core goal of pre-training is to give the model "prior knowledge" so that it has the basic ability to handle diverse tasks, similar to how humans accumulate common sense through extensive reading. The essence of pre-training is to use self-supervised learning to allow the large model to mine intrinsic correlations from massive unlabeled data and form transferable feature representations. In this embodiment, after pre-training, the initial state LLM large model has the general knowledge of knowledge in the field of education and teaching.

[0069] Step S204: Obtain question-setting instructions.

[0070] In this embodiment, if Figure 5 As shown, the question requirements entered by the user to the client ( Figure 5 The first content in the question) is used as a question-setting instruction.

[0071] Step S205: According to the question-setting instruction, the initial LLM large model generates an initial subjective question.

[0072] In this embodiment, the initial subjective question is generated by the initial state LLM large model according to the question setting requirements of the question setting instruction.

[0073] In an embodiment provided by the present invention, before obtaining the rating scale, the method further includes the following steps: Based on the content of the initial subjective questions and the answer requirements, a scoring table structure for the answer content is constructed, and the scoring table structure includes scoring standards and scoring levels; wherein the scoring standards are set according to the course objectives; and the scoring levels are set according to the graduation requirements.

[0074] In this embodiment, the scoring table structure is a tool for evaluating whether the answer content meets the requirements of the initial subjective questions and whether the expected training goals are achieved. Figure 3 As shown, the scoring criteria represent the specific aspects of the content evaluation. For example, the scoring criteria include content completeness, language expression, structure and organization, and innovation. The scoring level indicates the performance level of the content on a certain scoring criterion. For example, the scoring levels include excellent, good, fair, and poor, with each level indicating the upper limit of the score.

[0075] It should be noted that the scores listed on the grading scale represent the upper limit of the grade. For example, the upper limit of Excellent is 4 points, with a scoring range of 3 < Excellent ≤ 4; the upper limit of Good is 3 points, with a scoring range of 2 < Good ≤ 3; the upper limit of Fair is 2 points, with a scoring range of 1 < Fair ≤ 2; the upper limit of Poor is 1 point, with a scoring range of 0 < Poor ≤ 1. Scores can be displayed to two decimal places, for example, 0.78 points or 3.56 points.

[0076] In another embodiment of the present application, a scoring table structure for the initial subjective questions can also be constructed based on the specific content and requirements of the question-setting instructions, wherein the scoring table structure includes scoring criteria and scoring levels. In this case, the scoring table structure is a tool for evaluating whether the initial subjective questions meet the course objectives and whether they are consistent with the teaching content.

[0077] In an embodiment provided by the present invention, the above method further includes the following steps: Get evaluation response; Determining, based on the non-compliant evaluation information in the evaluation response, that the feedback quality of the first scoring evaluation information is low; If the feedback quality of the first scoring evaluation information is low, obtain a high-quality dataset. A high-quality dataset is a dataset composed of historical output meeting the standards, or a dataset specially constructed based on the requirements of the scoring scale; Based on the high-quality data set, the initial LLM large model is fine-tuned to obtain a fine-tuned LLM large model.

[0078] In this embodiment, the evaluation response indicates that the feedback quality of the first rating evaluation information is low. When the evaluation response is obtained, the client automatically marks the low quality indicator. Based on the evaluation response, the quality issue type is clarified. The quality issue types include missing scoring criteria for the initial subjective question, unclear scoring levels for the initial subjective question, missing scoring criteria for the answer content, and unclear scoring levels for the answer content.

[0079] For example, when the quality question type is an initial subjective question with an ambiguous rating for the language expression dimension, the rating scale module (Rubric) generates a fine-tuning instruction to "optimize the rating description for this dimension." The rating scale module transmits the fine-tuning instruction and the high-quality dataset to the fine-tuning module (LORA). Using low-rank matrix update technology, without changing the core parameters of the initial LLM large model, the parameter weights related to language expression scoring are specifically optimized, completing the precise calibration of the scoring logic and ensuring clearer and more objective rating determinations for this dimension in subsequent scoring. The low-rank matrix update technology uses processing methods well known in the art.

[0080] For example, when the quality question type is that the scoring criteria for the content completeness dimension of the answer content is missing, the scoring scale module (Rubric) generates a fine-tuning instruction to "supplement the scoring criteria description for this dimension". The scoring scale module transmits the fine-tuning instruction and the high-quality data set to the fine-tuning module (LORA). Through the low-rank matrix update technology, without changing the core parameters of the initial LLM large model, the parameter weights related to content completeness are supplemented in a targeted manner to complete the filling of the scoring criteria, ensuring that the grade judgment of this dimension is supported in subsequent scoring. In one embodiment provided by the present invention, the initial state LLM large model is fine-tuned based on the high-quality data set to obtain the fine-tuned LLM large model, including the following steps: Screening out text paragraphs containing descriptions of the causal relationship between the scoring criteria and the answer quality from the teaching knowledge base, extracting the teaching context information therein, manually identifying the causal relationship tree, and constructing a causal relationship dataset S; In the causal relationship dataset S, each data S={CTX, SD, QA, CRS}, where CTX={Location,Time}: CTX represents the context attribute, Location represents the place where the test was taken (e.g., Beijing, Shanghai, etc.), and Time represents the time when the test was taken (e.g., the year of the test, the semester, etc.); SD indicates discipline; QA refers to features related to the quality of answers, including standardization of answers, completeness of content, and logical coherence; CRS represents a causal relationship dataset, which includes cause variables and result variables; Provide contextual information for subjective questions based on contextual attributes, disciplines, and features related to answer quality. Use causal relationship datasets to reason and analyze subjective questions and construct question-answer pairs. Based on the question-answer pairs, the initial LLM large model is fine-tuned to obtain the fine-tuned LLM large model.

[0081] In an embodiment provided by the present invention, the above method further includes the following steps: Obtain the second scoring evaluation information of the answer content based on the fine-tuned LLM model; Sending the second rating evaluation information to the first terminal; or, Obtain the third-party scoring evaluation information of the answer content based on the fine-tuned LLM model; If it is determined that the feedback quality of the third scoring evaluation information is high, the third scoring evaluation information is sent to the first terminal.

[0082] In this embodiment, the fine-tuned LLM large model re-scores the answer content to obtain second score evaluation information, which is defaulted to high feedback quality and sent to the first terminal. Alternatively, in another embodiment, the fine-tuned LLM large model re-scores the answer content to obtain third score evaluation information, which is input into the adaptive module, which determines the feedback quality of the third score evaluation information. If the feedback quality of the third score evaluation information is determined to be high, the third score evaluation information is sent to the first terminal.

[0083] In an embodiment provided by the present invention, obtaining a dynamic smoothing coefficient according to the historical performance sequence includes the following steps: Establish a mapping relationship between the answer content and the scoring scale; Calculate the standard deviation of the historical performance series using the standard deviation algorithm; The dynamic smoothing coefficient is calculated according to the dynamic coefficient algorithm, wherein the dynamic coefficient algorithm is obtained by the following method:

[0084] in, represents the dynamic smoothing coefficient; represents the minimum smoothing factor; represents the maximum smoothing factor; Represents the standard deviation of the historical score series, where the historical score includes t items and t is a natural number (including 0); base represents the upper threshold of the score.

[0085] It should be noted that base represents the upper threshold of the score, which refers to the "normalized upper threshold" of the standard deviation of the historical score series of the LLM model. When ≥base, the system determines that the fluctuation of LLM large model scores has been large. reaches its maximum value. When the value exceeds the base, the upper limit is 1 after normalization.

[0086] In this embodiment, as shown in Table 1 and Table 2, the base initialization value is determined to be 0.3.

[0087]

[0088]

[0089] In this embodiment, the standard deviation can be the sample standard deviation. The standard deviation algorithm is obtained using the following method:

[0090] Represents various historical achievements, n represents the number of historical achievements, represents the sample mean.

[0091] For example, the values ​​of the historical performance sequence are 0.62, 0.80, 0.45, 0.92, and 0.68 respectively. =(0.62+0.80+0.45+0.92+0.68)÷5=0.694; =0.005476+0.011236+0.059536+0.051076+0.000196≈0.1275; , that is, the standard deviation is 0.18.

[0092] For example, The value is 0.18, the base value is 0.3, The value is 0.18, The value is 0.27, and the value of t is 4, then . And so on, ; ; ; .

[0093] In this step, according to the value changes of the historical performance sequence, Adaptive changes, i.e. The value of is dynamic and non-unique. It can quickly capture teaching changes and automatically adjust without manual intervention. It is dynamically adjusted according to the LLM model's scoring confidence, content innovation intensity, student fluctuation behavior, etc. More flexible control over the proportion of new and old information (training sample data). Therefore, based on dynamic extension points, it is suitable for adaptive update scenarios in teaching graph rubric generation and scoring.

[0094] In an embodiment provided by the present invention, obtaining the current matching degree value edge weight of the first rating evaluation information according to the historical score sequence and the dynamic smoothing coefficient includes the following steps: Calculate the residual initialization weight according to the residual weight algorithm; Calculate the current score contribution value according to the score contribution algorithm; The current matching degree value edge weight is calculated according to the matching strength algorithm, wherein the matching strength algorithm is obtained by the following method:

[0095] in, Indicates the current matching value edge weight; represents the sovereign control coefficient; Indicates the current score contribution value; represents the residual initialization weight; Represents the initial matching value edge weight.

[0096] The residual weight algorithm is obtained using the following method:

[0097] The score contribution algorithm is obtained using the following method:

[0098] It represents the score of the student's homework (i.e. the content of the answer) based on the tkth Rubric dimension (a certain scoring standard).

[0099] Indicates the half-life step of historical contribution and controls the range of historical scoring window. The following method is used to obtain:

[0100] It should be noted that: If the result of the calculation is not an integer, it is rounded down. For example, The calculated value is 2.3, which is rounded down to 2. The calculated value is 4.9, which is rounded down to 4. Rounding down ensures that the history window does not exceed the number of steps required for the half-life, which simplifies the calculation and avoids excessive accumulation of old scores. In addition, t starts at 0.

[0101] The fully expanded formula of the matching strength algorithm is:

[0102] This value is derived from the empirical principle of balancing "dynamic rubric scoring" and "original teaching structure design" in actual online learning systems (LMS). This value was set after a multi-dimensional analysis involving the following dimensions: Reference Dimension 1: From the perspective of actual teaching course design models (such as: Association of American Colleges and Universities' Valid Assessment of Learning in Undergraduate Education Rubrics, the full Chinese name is the American Association of Colleges and Universities' "Valid Assessment of Learning in Undergraduate Education Learning Outcomes" Rubric (VALUE Rubric) Grade Rubric Scheme; Bloom's Taxonomy (the full Chinese name is Bloom's Taxonomy of Cognitive Objectives) Grade Rubric Scheme), automatic scoring or dynamic scoring structures should not completely cover the teacher's original structure, and a certain degree of structural memory should be applied. Based on rubric modeling practice, it is recommended that: 70% scoring results are driven (emphasizing dynamic adaptation to student performance); 30% retain the original design structure (reflecting the general knowledge and stability of the course). Therefore, The default value of 0.7 is reasonable.

[0103] Reference Dimension 2: 30% of the teaching data requires innovative content, resulting in significant noise. However, within the teaching vertical, 70% of the data has a long-term stable preference. A stratified range of 0.2 < α ≤ 0.35 (indicating 20% ​​to 35% of the data is historically contributed old information) is set to accommodate long-term preferences while smoothing short-term fluctuations. Nearly 70% to 80% of the data favors current new information observations. This is mapped to the structure of the self-developed formula, with λ = 0.7, which conforms to the empirical principle of "current dominance, historical stability" for smooth integration.

[0104] Reference Dimension 3: When a general model is fine-tuned and used for automatic rubric scoring, the model itself has a certain risk of score drift or student "sudden performance" phenomenon.

[0105] Training performance after using synchronized values: λ=1.0: completely controlled by the score, high risk; λ=0.5: slow response to score changes; λ=0.7: Achieves a practical balance between "responsiveness" and "structural stability"; the structure is relatively stable, allowing dynamic scoring to adjust the direction.

[0106] Conclusion The numerical strategy of λ is adopted when refining the scene: λ=0.7 is the default value that strikes a balance between responsiveness and structural memory in teaching evaluation. The system also dynamically adjusts λ based on the teacher's preferences, by passing parameters to adjust λ, and by monitoring the confidence level of the large model's scores. This achieves the goal of an automatic rolling regression with flexible and dynamic adjustment strategies.

[0107] also, It can also be set to other values ​​according to other application scenarios and application requirements. Example

[0108] The following describes how to determine the feedback quality of the first scoring evaluation information based on a scoring standard dimension (e.g., content integrity). In this embodiment, as shown in Table 3, the values ​​of various parameters are taken to calculate the edge weight of the current matching degree. Here, t represents a discrete time step. For example, the most recent score before the time indicated by the timestamp in the first scoring evaluation information can be used as a historical score. Alternatively, the most recent five scores before the time indicated by the timestamp in the first scoring evaluation information can be used as historical scores to construct a historical score sequence.

[0109]

[0110] For example, when t=4 (t starts at 0), under the same scoring criteria, The scores are shown in Table 4.

[0111] , the value is the largest, and the value is 3. The value is taken as 2, so in the following summation formula, k = 0, 1, 2: ; From this calculation we can get: .

[0112] In an embodiment provided by the present invention, obtaining a confidence value of the first scoring evaluation information according to the edge weight of the current matching degree; and obtaining feedback quality of the first scoring evaluation information according to the confidence value include the following steps: The confidence value of the first scoring evaluation information is calculated according to a confidence algorithm, wherein the confidence algorithm is obtained by the following method:

[0113] Where Z represents the confidence value; a first score representing the first scoring evaluation information; Indicates the expected score. The first score of the first rating evaluation information refers to the score under a certain rating standard.

[0114] Determining whether the confidence value falls within a confidence interval; If the confidence value falls within the confidence interval, it is determined that the feedback quality of the first scoring evaluation information is high.

[0115] It should be noted that the confidence value is calculated for a specific scoring criterion; high feedback quality is a quality confirmation for one of the above scoring criteria. In this application, the comprehensive scoring and evaluation information can be sent to the first terminal only when the feedback quality of the scoring and evaluation information corresponding to each scoring criterion is high under the four scoring criteria; or the scoring and evaluation information corresponding to a specific scoring criterion is sent to the first terminal only when the feedback quality of the scoring and evaluation information is high.

[0116] The expected score is calculated according to a scoring algorithm, which is obtained by the following method: ; Indicates the maximum score possible for a particular grading criterion.

[0117] For example, 30 minutes, is 0.325, then =0.325×30=9.75; Taking the value of 9.9 (the actual score this time), Z = (9.9 - 9.75) ÷ 0.18 = 0.83. The confidence interval is the range of values ​​from "-1.645 (inclusive) to 1.645 (inclusive)." Determining whether the confidence value falls within the confidence interval means that if |Z| ≤ 1.645, the confidence value is considered to be within the confidence interval. Since |0.83| < 1.645, it can be determined that the confidence value falls within the confidence interval, and the feedback quality of the first rating evaluation information is determined to be high, that is, the rating accuracy is above 90%. In this step, the confidence interval can be set. For example, the confidence interval can be within the range of values ​​from "-1.512 (inclusive) to 1.512 (inclusive)", and so on.

[0118] In the existing technology, the noise of rating data is about 40%. Through the technical solution of this application, using the various steps in the above method, the noise of the generated rating evaluation information is controlled to less than 10%, and the rating accuracy is above 90%.

[0119] Example 2 In the embodiments of this application, the scoring and evaluation information for each topic (answer content or subjective question) corresponds to multiple different scoring criteria dimensions. This is explained using the example of "assessing the overall feedback quality of the scoring and evaluation information based on four scoring criteria dimensions (e.g., content completeness, language expression, structure, and innovation) for the answer content."

[0120] Step S31: Calculate and determine the feedback quality of the scoring evaluation information corresponding to each scoring standard dimension in sequence. If the feedback quality of the scoring evaluation information for all four scoring standard dimensions is low, the combined scoring evaluation information is determined to be non-compliant and transmitted to the scoring scale. Fine-tune the initial LLM model. The fine-tuning process is consistent with the description in the article and will not be repeated here.

[0121] Step S32: When the rating evaluation information of at least one rating standard dimension is of high feedback quality, a confidence value Z' of the comprehensive rating evaluation information is calculated according to a comprehensive algorithm. The confidence value Z' of the comprehensive rating evaluation information is obtained by the following method: ; Among them, Z' represents the confidence value of the comprehensive rating evaluation information; a represents the total adjustment coefficient, 0<a<1; M h Represents the sum of the confidence values ​​of each rating standard dimension with high feedback quality; M l Indicates the sum of the confidence values ​​of each rating scale dimension where the feedback quality is low.

[0122] Step S33: Compare Z' with the comprehensive threshold. When Z' is greater than the comprehensive threshold, the feedback quality of the comprehensive rating and evaluation information is determined to be high, and the rating and evaluation information for all four rating standard dimensions is output. When Z' is less than the comprehensive threshold, the feedback quality of the comprehensive rating and evaluation information is determined to be low, and the rating and evaluation information for all four rating standard dimensions is not output, and the process returns to fine-tuning. The comprehensive threshold can be the average of the confidence values ​​Z obtained historically that meet the high-quality standard, or another set value.

[0123] In an embodiment of the present application, a can be dynamically adjusted according to the requirements of student abilities based on the content of the answers. When students' abilities with high confidence are given more importance, the confidence corresponding to the scoring and evaluation information generated by a single item or several items under the corresponding scoring standard dimension is high (as long as the confidence corresponding to the scoring and evaluation information generated under the corresponding scoring standard dimension is high), which can make up for the impact of the low confidence corresponding to the scoring and evaluation information generated under other scoring standard dimensions of the students on the feedback quality. The value range of a is set to 0.7<a<1. Increase the proportion of scoring standard dimensions with high feedback quality, and move closer to the high feedback quality of the comprehensive scoring and evaluation information. Among them, high confidence means that the confidence value Z corresponding to the scoring and evaluation information is within the confidence interval; low confidence means that the confidence value Z corresponding to the scoring and evaluation information is not within the confidence interval.

[0124] When answer content prioritizes students' abilities in high-confidence items, the confidence level of the scoring and evaluation information generated for a single item or several items under the corresponding scoring criteria dimensions should be high. (This can only be achieved when the confidence level of the scoring and evaluation information generated under the corresponding scoring criteria dimensions reaches a certain proportion (for example, the confidence level of three scoring criteria dimensions is high)). This can offset the impact of low confidence levels on feedback quality for students' other scoring criteria dimensions. Setting the value range of a to 0<a<0.3 reduces the proportion of scoring criteria dimensions with high feedback quality. This can only compensate for the impact of low-confidence feedback quality when the abilities in high-confidence items are outstanding. This can be used to select answer content that requires a higher level of a certain ability.

[0125] In the case where the content of the answer is more important for the comprehensive scoring criteria of the student, for example, the value range of a is set to 0.3<a<0.7. Only when the score with low confidence in the scoring criteria dimension corresponding to the answer content is close to the confidence level that meets the requirements can the impact of low confidence corresponding to the scoring evaluation information generated under other scoring criteria dimensions of the student on the score be compensated, reflecting the reconciliation effect of scoring criteria dimensions with high feedback quality and scoring criteria dimensions with low feedback quality. Among them, the score with low confidence in the scoring criteria dimension corresponding to the answer content is close to the confidence level that meets the requirements, which means that the difference between the low confidence and the confidence level that meets the requirements of the scoring criteria dimension corresponding to the answer content is less than the difference threshold, which can be set as needed.

[0126] In one embodiment of the present application, a can be set based on the ability requirements of the answer content, or can be prioritized based on analyzing historical data and divided into three categories. The characteristics of each category of answer content are extracted based on the large model. When new answer content is obtained, the characteristics of the new answer content are compared with the characteristics of each category of answer content to automatically obtain the corresponding value of a. This is not limited in the embodiments of the present application.

[0127] Thus, in the embodiments of the present application, based on the characteristics of each type of answer content, the answer content corresponding to the comprehensive scoring and evaluation information that meets the feedback quality can be determined, and students who meet the requirements can be selected from the answer content for subsequent training, rather than simply cultivating students' general abilities for different subjective questions. This is more conducive to the development and cultivation of students' specialties.

[0128] In one embodiment of the present invention, after pre-training the large language model based on the teaching knowledge base to obtain the initial state LLM large model, the method provided by the present invention performs the following steps: Constructing a multidimensional feature vector set based on the teaching knowledge base, wherein the multidimensional feature vector set includes a dimension coverage vector, a standard description vector, and a domain adaptation vector; Specifically, constructing a multidimensional feature vector set based on the teaching knowledge base refers to starting from the core dimensions of the knowledge system in the teaching knowledge base, such as the structure, description standardization and domain adaptability, through systematic feature extraction and quantitative modeling. The dimension coverage vector refers to extracting structural features such as the hierarchical relationship, logical context and coverage of the knowledge modules in the teaching knowledge base to form a vector that represents the dimensional completeness of the knowledge system. The standard description vector refers to extracting features from the standardization, process and practical guidance of the knowledge description to form a vector that measures the quality of knowledge representation. The domain adaptation vector refers to extracting the adaptability characteristics of the knowledge content based on the needs, scenario characteristics and audience characteristics of the target domain to form a domain-customized vector.

[0129] Furthermore, the output structure rationality scoring standard is set according to the dimension coverage vector; the operability scoring standard is set according to the standard description vector; and the domain adaptability scoring standard is set according to the domain adaptation vector. The question scoring value R is calculated according to the question scoring algorithm, wherein the question scoring algorithm is obtained by the following method: R= *Structural rationality score+ *Operability score+ *Domain suitability score; in + + =1, and 、 、 Dynamically adjust according to the application scenario, among which, 、 、 Both represent rating coefficients.

[0130] Based on the multi-dimensional feature vector, the scoring criteria for structural rationality, operability and domain adaptability are output.

[0131] According to the formula R= *Structural rationality score+ *Operability score+ *Domain adaptability score, calculate the question score; + + =1, and 、 、 Dynamically adjust according to the application scenario. The technologies used here: Transformer architecture, attention mechanism, and positional encoding ensure the ability to understand the logic of rubric language.

[0132] It should be noted that in this application, you can also construct a topic containing Figure 3 The scoring criteria and scoring dimensions are shown in the scoring scale. Based on the scoring scale of the questions, the initial subjective questions generated by the initial LLM large model are scored.

[0133] In an embodiment provided by the present invention, the above method further includes the following steps: Preprocessing the initial subjective questions, the answer content, and the first scoring evaluation information with high feedback quality to construct a second teaching knowledge base, wherein the preprocessing includes text cleaning, text deduplication, and text segmentation; Based on the second teaching knowledge base, pre-training is performed on the initial state LLM large model to obtain an advanced version of the LLM large model.

[0134] In this step, the initial LLM model is pre-trained based on the second teaching knowledge base to obtain an advanced LLM model. This advanced LLM model strengthens the ability to reason about knowledge points, optimizes the learning of the "question-evaluation" logic, and corrects the random deviations in the initial LLM model's question design and difficulty control.

[0135] like Figure 4 As shown, the technical solution of the present invention builds an intelligent scoring system based on the pre-trained initial state LLM large model, and realizes the automation and continuous optimization of question setting and scoring through the following steps: Step A: The system first receives the teacher's input for setting questions, including parameters such as the scope of knowledge points, difficulty level, and question type requirements. The pre-trained initial LLM model generates the initial subjective questions and a corresponding scoring scale based on the teacher's input. The scoring scale is used to score the answers to the initial subjective questions. The scoring scale adopts a multi-dimensional design, such as language expression, structural organization, and content completeness. Grading standards and anchor examples are defined for each dimension to form a structured scoring framework.

[0136] Step B: Validate the rating scale. This involves multi-dimensional testing to confirm whether the rating scale effectively achieves the rating objectives and avoids subjective bias or loopholes. Examples of this validation include discrimination testing (using a test set to verify whether the rating scale effectively distinguishes between responses of varying quality); consistency assessment (using metrics like the Kappa coefficient to verify the consistency of multiple users using the rating scale); and anti-interference verification (implanting pre-set biased answers (e.g., text with clear structure but empty content) to verify the robustness of the rating scale). If the validation fails, the system will flag the suspicious dimension and trigger a second calibration by the teacher until the rating scale passes validation.

[0137] Step C: Based on the teacher's input, a scoring prompt is generated, and the initial LLM model performs the scoring. For example, after the teacher enters a scoring prompt, the system converts it into a scoring prompt template that the LLM model can understand. The template includes information such as the answer content, scoring dimensions, assessment focus, and scoring criteria. Based on this prompt, the initial LLM model scores the answer content and outputs a structured result, including scores for each dimension, textual feedback, and suggestions for improvement.

[0138] Step D: Input the scoring evaluation information generated by the initial LLM large model into the adaptive module, calculate the confidence value by the adaptive module, and determine the feedback quality of the scoring evaluation information based on the confidence value.

[0139] Step E: If the feedback quality is high, the rating and evaluation information is entered into the teaching knowledge base. This information is used as part of the teaching text data and pre-trained into the initial LLM model. In this step, the rating and evaluation information that passes quality verification is stored in the teaching knowledge base, which is indexed by subject, knowledge point, and other dimensions. Exemplarily, the system employs an incremental pre-training strategy, regularly extracting data from the knowledge base to continuously train the initial LLM model, improving its performance in specific domain-specific rating tasks.

[0140] Step F: If the feedback quality is low, determine the scoring criteria corresponding to the rating evaluation information. Send the low feedback quality information based on the scoring criteria to the rating scale. The rating scale generates fine-tuning instructions, which are sent to the fine-tuning module. The fine-tuning module fine-tunes the initial LLM model to obtain a fine-tuned LLM model. Using the fine-tuned LLM model, regenerate the rating evaluation information based on the aforementioned scoring criteria.

[0141] In the second aspect, the embodiment of the present invention provides a system for enhancing the credibility of teaching evaluation in an education model, such as Figure 6 As shown, the system includes an acquisition unit 601, a quality evaluation unit 602 and a quality feedback unit 603; The acquisition unit 601 is used to acquire a scoring prompt, and acquire first scoring evaluation information of the answer content according to the scoring prompt; The quality evaluation unit 602 is configured to obtain an interactive object corresponding to the first rating evaluation information, and construct different users' scores into a historical score sequence under the same rating standard; obtain a dynamic smoothing coefficient based on the historical score sequence; obtain a current matching numerical edge weight of the first rating evaluation information based on the historical score sequence and the dynamic smoothing coefficient; obtain a confidence value of the first rating evaluation information based on the current matching numerical edge weight; and obtain feedback quality of the first rating evaluation information based on the confidence value; The quality feedback unit 603 is configured to determine that the first scoring and evaluation information meets the requirements when the feedback quality of the first scoring and evaluation information is high, and send the first scoring and evaluation information to the first terminal.

[0142] In one embodiment provided by the present invention, the system further includes a pre-training unit and a question setting unit; The pre-training unit is configured to obtain teaching text data from an online teaching system; pre-process the obtained teaching text data to construct a teaching knowledge base, wherein the pre-processing includes text cleaning, text deduplication, and text segmentation; and pre-train a large language model based on the teaching knowledge base to obtain an initial state LLM large model. The question-setting unit is used to obtain a question-setting instruction; according to the question-setting instruction, the initial state LLM large model generates an initial subjective question.

[0143] In a third aspect, an embodiment of the present invention provides an electronic device, such as Figure 7 As shown, the electronic device 70 includes: a memory 701, a processor 702, and a computer program stored in the memory 701 and executable on the processor 702. When the processor 702 executes the computer program, the steps of the above method are implemented.

[0144] The electronic device 70 may be a computing device such as a tablet computer, a desktop computer, or a cloud server. The electronic device 70 may include but is not limited to a processor 702 and a memory 701. It will be understood by those skilled in the art that Figure 7 This is merely an example of the electronic device 70 and does not constitute a limitation on the electronic device 70 . The electronic device 70 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0145] The processor 702 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. The processor 702 may also be a GPU / DCU computing processor.

[0146] In some embodiments, the memory 701 may be an internal storage unit of the electronic device 70, such as a hard drive or memory of the electronic device 70. In other embodiments, the memory 701 may also be an external storage device of the electronic device 70, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 70. Furthermore, the memory 701 may include both an internal storage unit of the electronic device 70 and an external storage device. The memory 701 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 701 may also be used to temporarily store data that has been output or is about to be output.

[0147] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which depends on the functions involved.

[0148] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0149] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application by those skilled in the art should be included within the scope of protection of this application.

Claims

1. A method for enhancing the credibility of teaching evaluation in an educational macro model, characterized in that: The method comprises: Obtaining a scoring prompt, and obtaining first scoring evaluation information of the answer content according to the scoring prompt; Obtaining the interactive object corresponding to the first rating evaluation information, and constructing the scores of different users into a historical score sequence under the same rating standard; Obtaining a dynamic smoothing coefficient based on the historical performance sequence; Obtaining a current matching degree value edge weight of the first scoring evaluation information according to the historical score sequence and the dynamic smoothing coefficient; Obtaining a confidence value of the first rating evaluation information according to the current matching degree numerical edge weight; Obtaining feedback quality of the first scoring evaluation information according to the confidence value; If the feedback quality of the first scoring evaluation information is high, it is determined that the first scoring evaluation information meets the requirements, and the first scoring evaluation information is sent to the first terminal.

2. The method according to claim 1, characterized in that Before obtaining the scoring prompt and obtaining first scoring evaluation information of the answer content according to the scoring prompt, the method further includes: Obtain teaching text data of the online teaching system; Preprocessing the acquired teaching text data to construct a teaching knowledge base, wherein the preprocessing includes text cleaning, text deduplication and text segmentation; Based on the teaching knowledge base, pre-training is performed on the large language model to obtain an initial state LLM large model; Get the question setting instructions; According to the question-setting instruction, the initial subjective question is generated by the initial state LLM large model.

3. The method according to claim 1, characterized in that Before obtaining the scoring prompt and obtaining first scoring evaluation information of the answer content according to the scoring prompt, the method further includes: Obtaining an initial subjective question and a rating scale corresponding to the answer content; wherein the answer content is the answers of multiple users to the initial subjective question; The step of obtaining a scoring prompt and obtaining first scoring evaluation information of the answer content according to the scoring prompt includes: Obtaining the scoring prompt and the scoring scale; The scoring prompt, the scoring scale, and the answer content are input into the initial LLM macro model to obtain the first scoring evaluation information of the answer content.

4. The method according to claim 2, characterized in that The method further comprises: When the feedback quality of the first scoring evaluation information is low, obtaining a high-quality data set; Based on the high-quality data set, the initial LLM large model is fine-tuned to obtain a fine-tuned LLM large model.

5. The method according to claim 4, characterized in that Fine-tuning the initial LLM large model based on the high-quality data set to obtain a fine-tuned LLM large model includes: Screening out text paragraphs containing descriptions of the causal relationship between the scoring criteria and the quality of the answer from the teaching knowledge base; Constructing a causal relationship dataset based on the scoring criteria and the causal relationship description of the answer quality in the text paragraph; Using the causal relationship dataset to reason and analyze the input subjective questions, and construct question-answer pairs; Based on the question-answer pair, the initial LLM large model is fine-tuned to obtain the fine-tuned LLM large model.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining second scoring evaluation information of the answer content based on the fine-tuned LLM model; Sending the second rating evaluation information to the first terminal; or, Obtaining third scoring evaluation information of the answer content based on the fine-tuned LLM model; If it is determined that the feedback quality of the third scoring evaluation information is high, the third scoring evaluation information is sent to the first terminal.

7. The method according to claim 3, characterized in that The step of obtaining a dynamic smoothing coefficient based on the historical performance sequence includes: Calculate the standard deviation of the historical performance series; The dynamic smoothing coefficient is calculated according to a dynamic coefficient algorithm, wherein the dynamic coefficient algorithm is obtained by the following method: ; in, represents the dynamic smoothing coefficient; represents the minimum smoothing factor; represents the maximum smoothing factor; Represents the standard deviation of the historical score series; base represents the upper threshold of the score.

8. The method according to claim 1 or 7, characterized in that The step of obtaining the current matching degree value edge weight of the first scoring evaluation information according to the historical score sequence and the dynamic smoothing coefficient includes: Calculate the residual initialization weight according to the residual weight algorithm; Calculate the current score contribution value according to the score contribution algorithm; The current matching degree value edge weight is calculated according to the matching strength algorithm, wherein the matching strength algorithm is obtained by the following method: ; in, Indicates the current matching value edge weight; represents the sovereign control coefficient; Indicates the current score contribution value; represents the residual initialization weight; Represents the initial matching value edge weight.

9. The method according to claim 8, characterized in that obtaining a confidence value of the first rating evaluation information according to the edge weight of the current matching degree value; Obtaining feedback quality of the first scoring evaluation information according to the confidence value includes: The confidence value of the first scoring evaluation information is calculated according to a confidence algorithm, wherein the confidence algorithm is obtained by the following method: ; Among them, Z represents the confidence value; a first score representing the first scoring evaluation information; represents the expected score; Determining whether the confidence value falls within a confidence interval; If the confidence value falls within the confidence interval, it is determined that the feedback quality of the first scoring evaluation information is high.

10. The method according to claim 2, characterized in that After pre-training the large language model based on the teaching knowledge base to obtain the initial state LLM large model, the method further includes: Constructing a multidimensional feature vector set based on the teaching knowledge base, wherein the multidimensional feature vector set includes a dimension coverage vector, a standard description vector, and a domain adaptation vector; Setting the output structure rationality scoring standard according to the dimension coverage vector; setting the operability scoring standard according to the standard description vector; setting the domain adaptability scoring standard according to the domain adaptation vector; The question scoring value R is calculated according to the question scoring algorithm, wherein the question scoring algorithm is obtained by the following method: R= *Structural rationality score+ *Operability score+ *Domain suitability score; in, + + =1, and 、 、 Dynamically adjust according to application scenarios, 、 、 Both represent rating coefficients.

11. The method according to claim 1, wherein The method further comprises: Preprocessing the initial subjective questions, the answer content, and the first scoring evaluation information with high feedback quality to construct a second teaching knowledge base, wherein the preprocessing includes text cleaning, text deduplication, and text segmentation; Based on the second teaching knowledge base, pre-training is performed on the initial state LLM large model to obtain an advanced version of the LLM large model.

12. A system for enhancing the credibility of teaching evaluation in an educational model, characterized in that: The system includes an acquisition unit, a quality assessment unit and a quality feedback unit; The obtaining unit is configured to obtain a scoring prompt, and obtain first scoring evaluation information of the answer content according to the scoring prompt; The quality evaluation unit is configured to obtain an interactive object corresponding to the first rating evaluation information, and construct the scores of different users into a historical score sequence under the same rating standard; obtain a dynamic smoothing coefficient based on the historical score sequence; obtain a current matching numerical edge weight of the first rating evaluation information based on the historical score sequence and the dynamic smoothing coefficient; obtain a confidence value of the first rating evaluation information based on the current matching numerical edge weight; and obtain feedback quality of the first rating evaluation information based on the confidence value; The quality feedback unit is configured to determine that the first scoring and evaluation information meets the requirements when the feedback quality of the first scoring and evaluation information is high, and send the first scoring and evaluation information to the first terminal.

13. The system according to claim 12, wherein: The system also includes a pre-training unit and a question-setting unit; The pre-training unit is configured to obtain teaching text data from an online teaching system; pre-process the obtained teaching text data to construct a teaching knowledge base, wherein the pre-processing includes text cleaning, text deduplication, and text segmentation; and pre-train a large language model based on the teaching knowledge base to obtain an initial state LLM large model. The question-setting unit is used to obtain a question-setting instruction; according to the question-setting instruction, the initial state LLM large model generates an initial subjective question.

14. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 11 when executing the computer program.

Images