Method and device for automatically scoring short answer based on large language model

Abstract

The invention relates to the technical field of education, and discloses a simple answer automatic scoring method and device based on a large language model. The simple answer automatic scoring method based on the large language model comprises the steps of obtaining question information, score information, reference answer information and simple answer information; extracting a plurality of key points in the reference answer information, and averagely distributing scores to each key point; a large language model is called to evaluate the short answer information according to the key points, whether the short answer information covers the key points or not is judged one by one according to the key points, corresponding binary feedback is generated, and detailed evaluation information is generated; generating the confidence coefficient of the answer information of the short answer, sending the answer information of the short answer with the confidence coefficient lower than a threshold value to a manual scoring intervention end, obtaining scoring result information, and dynamically adjusting cue words according to the scoring result information; the scoring result information or the detailed evaluation information is sent to the student terminal; the method has the advantage of improving scoring accuracy and scoring consistency.

Description

Technical Field

[0001] The present invention relates to the field of educational technology, and in particular, to a method and device for automatically grading short-answer questions based on a large language model. Background Art

[0002] With the development of educational technology, automatic grading systems are increasingly widely used in educational assessment. However, the automatic grading of short-answer questions still faces many challenges. Traditional automatic grading methods usually rely on keyword matching or rule-based systems. However, due to the complexity and ambiguity of natural language, these methods perform poorly in dealing with synonyms, paraphrases, and implicit meanings, resulting in inaccurate grading. In addition, existing automatic grading systems often lack fine-grained feedback, making it difficult to help students understand their mistakes and improve.

[0003] In recent years, large language models (LLMs) have made remarkable progress in the field of natural language processing, especially in understanding context and semantics. However, the application of existing LLMs in automatic grading still has limitations. Especially when dealing with vague or incomplete student answers, the consistency and accuracy of grading still need to be improved.

[0004] Therefore, a method and device for automatically grading short-answer questions based on a large language model are provided to solve the above problems. Summary of the Invention

[0005] The main object of the present invention is to solve the problems that traditional automatic grading methods rely on keyword matching and rule systems, resulting in poor performance in dealing with synonyms, paraphrases, and implicit meanings, inaccurate grading, and lack of fine-grained feedback, making it difficult to help students understand their mistakes and improve.

[0006] The first aspect of the present invention provides a method for automatically grading short-answer questions based on a large language model. The method for automatically grading short-answer questions based on a large language model includes:

[0007] Obtain short-answer question information, score information, reference answer information, and short-answer question answer information, that is, the answer information of the short-answer questions sent by students;

[0008] Extract the key delimiters in the reference answer information, decompose the reference answer information into several key points according to the key delimiters, evenly distribute the scores to each key point according to the score information and the key points, or directly obtain the manual key point division information and score distribution information, or extract the key delimiters in the reference answer information, decompose the reference answer information into several key points, send the key point information to the manual scoring end, and obtain the manual scoring score information. Any one of the above three operation methods can be selected;

[0009] Call a large language model to evaluate the short answer question response information based on key points, judge whether the short answer question response information covers each key point one by one according to the key points and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback;

[0010] Generate the confidence level of the short answer question response information, send the response information with a confidence level lower than the threshold to the manual scoring intervention end, obtain the scoring result information sent by the manual intervention end, and dynamically adjust the prompt words according to the scoring result information;

[0011] Send the scoring result information or the detailed evaluation information to the student end.

[0012] Furthermore, before generating detailed evaluation information for each binary feedback, it also includes:

[0013] Filter non-academic language through a keyword filter, and the non-academic language includes emotional appeal language and role-playing language.

[0014] Furthermore, the extraction of the key delimiters in the reference answer information, the decomposition of the reference answer information into several key points according to the key delimiters, and the average distribution of the scores to each key point according to the score information and key points include:

[0015] Extract the key delimiters in the reference answer information, and the key delimiters include semicolon delimiters and period delimiters;

[0016] Take the paragraph before the first key delimiter as one key point, and take the paragraphs between every two delimiters as one key point respectively;

[0017] Obtain the score information and evenly distribute the scores to each key point.

[0018] Furthermore, the call of the large language model to evaluate the short answer question response information based on key points, the judgment of whether the short answer question response information covers each key point one by one according to the key points and the generation of corresponding binary feedback, and the generation of detailed evaluation information for each binary feedback include:

[0019] Call the Llama-3.1-70B large language model to evaluate the short answer question information;

[0020] Judge whether the short answer question response information covers each key point one by one according to the key points. If it covers, generate the binary feedback as True feedback. If it does not cover, generate the binary feedback as False feedback;

[0021] Generate detailed evaluation information for each binary feedback, and at the same time assign scores to the short answer question response information.

[0022] Further, the sending of the scoring result information or the detailed evaluation information to the student side includes:

[0023] Determine whether the confidence level is lower than the threshold. If so, send the scoring result information to the student side. If not, send the detailed evaluation information to the student side.

[0024] The second aspect of the present invention provides a short-answer question automatic scoring device based on a large language model. The short-answer question automatic scoring device based on a large language model includes:

[0025] An information acquisition module, configured to acquire question information, score information, reference answer information, and short-answer question response information;

[0026] A key point extraction and scoring module, configured to extract key delimiters in the reference answer information, decompose the reference answer information into several key points according to the key delimiters, and evenly distribute the scores to each key point according to the score information and the key points;

[0027] A detailed scoring module, configured to call the large language model to evaluate the short-answer question response information according to the key points, judge one by one whether the short-answer question response information covers the key point and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback;

[0028] A prompt word dynamic adjustment module, configured to generate the confidence level of the short-answer question response information, send the answer information with a confidence level lower than the threshold to the manual scoring intervention side, obtain the scoring result information sent by the manual intervention side, and dynamically adjust the prompt word according to the scoring result information;

[0029] A scoring result sending module, configured to send the scoring result information or the detailed evaluation information to the student side.

[0030] The information acquisition module includes a question information acquisition unit, a score information acquisition unit, a reference answer information acquisition unit, and a short-answer question response acquisition unit;

[0031] The question information acquisition unit is configured to acquire question information;

[0032] The score information acquisition unit is configured to acquire score information;

[0033] The reference answer information acquisition unit is configured to acquire reference answer information;

[0034] The short-answer question response acquisition unit is configured to acquire short-answer question response information.

[0035] The key point extraction and scoring module includes: a key delimiter extraction unit and a scoring unit;

[0036] A key separator extraction unit is used to extract key separators in the reference answer information. The key separators include semicolon separators and period separators. The paragraph before the first key separator is regarded as a key point, and the paragraphs between every two separators are regarded as key points respectively.

[0037] A scoring unit is used to obtain score information and evenly distribute the scores to each key point.

[0038] A third aspect of the present invention provides an electronic device, which includes a memory and at least one processor. Instructions and data are stored in the memory.

[0039] The at least one processor invokes the instructions and data in the memory so that the electronic device executes each step of the short answer question automatic scoring method based on the large language model as described above.

[0040] A fourth aspect of the present invention provides a readable storage medium, on which instructions and data are stored. When the instructions are executed by a processor, each step of the short answer question automatic scoring method based on the large language model as described above is implemented.

[0041] The present invention provides a short answer question automatic scoring method based on the large language model. By introducing key point scoring and dynamic adjustment of prompt words, the accuracy, consistency and feedback quality of scoring are significantly improved. It can improve the fairness and transparency of scoring. Through the anti-misleading mechanism, it effectively prevents the manipulation of the scoring system through non-academic language and ensures the fairness of scoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of the short answer question automatic scoring method based on the large language model provided by the present invention.

[0043] Figure 2 It is a schematic diagram of prompt design provided by the present invention.

[0044] Figure 3 It is a schematic diagram of the structure of the short answer question automatic scoring device based on the large language model provided by the present invention.

[0045] Figure 4 It is a schematic diagram of the structure of the electronic device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] An embodiment of the present invention provides a method for automatically grading short-answer questions based on a large language model, which includes obtaining question information, score information, reference answer information, and short-answer question response information; extracting key delimiters in the reference answer information, decomposing the reference answer information into several key points according to the key delimiters, and evenly distributing the scores to each key point according to the score information and the key points; calling the large language model to evaluate the short-answer question response information according to the key points, judging one by one whether the short-answer question response information covers the key point and generating corresponding binary feedback, and generating detailed evaluation information for each binary feedback; generating the confidence level of the short-answer question response information, sending the short-answer question response information with a confidence level lower than the threshold to the manual grading intervention terminal, obtaining the score result information sent by the manual intervention terminal, and dynamically adjusting the prompt words according to the score result information; sending the score result information or the detailed evaluation information to the student terminal. The main purpose of the present invention is to solve the problems that the traditional automatic grading method relies on keyword matching and rule systems, resulting in poor performance, inaccurate grading when dealing with synonyms, paraphrasing, and implicit meanings, and lack of fine-grained feedback, which is difficult to help students understand mistakes and improve.

[0047] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of the present invention are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order different from that shown or described herein. In addition, the terms "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] For ease of understanding, the specific process of the embodiment of the present invention is described below. Please refer to Figures 1-3 The first embodiment of the method for automatically grading short-answer questions based on a large language model provided by the present invention includes:

[0049] Obtain question information, score information, reference answer information, and short-answer question response information;

[0050] Extract the key delimiters in the reference answer information, decompose the reference answer information into several key points according to the key delimiters, and evenly distribute the scores to each key point according to the score information and the key points;

[0051] Specifically, it includes:

[0052] Extract the key delimiters in the reference answer information, where the key delimiters include semicolon delimiters and period delimiters;

[0053] Take the paragraph before the first key delimiter as one key point, and take the paragraphs between every two delimiters as one key point respectively;

[0054] Obtain the score information and evenly distribute the scores to each key point; each key point is marked with a tag in the format of <mark>< / mark> , where the content inside the tag represents the content of the key point, and Mark represents the score of this key point.

[0055] Filter out the non-academic language in the short answer question response information through a keyword filter, where the non-academic language includes emotional appeal language and role-playing language;

[0056] Call a large language model to evaluate the short answer question response information according to the key points, judge whether the short answer question response information covers this key point one by one according to the key points and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback;

[0057] Specifically include:

[0058] Call the Llama-3.1-70B large language model to evaluate the short answer question information;

[0059] Judge whether the short answer question response information covers this key point one by one according to the key points. If it covers, generate the binary feedback as True feedback. If it does not cover, generate the binary feedback as False feedback;

[0060] Generate detailed evaluation information for each binary feedback to explain the correctness or deficiencies of the student's answer, and at the same time assign scores to the short answer question response information.

[0061] Generate the confidence level of the short answer question response information. Send the answer information with a confidence level lower than the threshold to the manual scoring intervention terminal, obtain the scoring result information sent by the manual intervention terminal, and dynamically adjust the prompt words according to the scoring result information;

[0062] Send the scoring result information or detailed evaluation information to the student terminal;

[0063] Specifically include judging whether the confidence level is lower than the threshold. If so, send the scoring result information to the student terminal. If not, send the detailed evaluation information to the student terminal.

[0064] In the process of specifically applying to the scoring of computer science short answer questions:

[0065] Obtain the question. The specific question content is: Please explain the time complexity of the quicksort algorithm and illustrate its time complexity in the worst case.

[0066] Obtain the reference answer, and the specific content of the reference answer is:

[0067] <3>The average time complexity of quicksort is O(n log n). <!--3-->

[0068] <2>The time complexity in the worst case is O(n^2), which occurs when the selected pivot element is always the smallest or largest element. <!--2-->

[0069] Obtain the student's answer, that is, the short-answer question answer information, and the specific content is:

[0070] The average time complexity of quicksort is O(n log n), and in the worst case it is O(n^2).

[0071] The specific scoring process is as follows:

[0072] The model first decomposes the reference answer into two key points.

[0073] For key point <3> <!--3-->, the model determines that the student's answer mentions "the average time complexity is O(n log n)", so it gives a judgment of "True".

[0074] For key point <2> <!--2-->, the model determines that the student's answer mentions "in the worst case it is O(n^2)", so it gives a judgment of "True".

[0075] The model generates feedback, indicating that the student's answer is completely correct and explains the content of each key point.

[0076] Prompt dynamic adjustment mechanism:

[0077] If the model has low confidence in the judgment of a certain key point, the system will request manual intervention.

[0078] After manual scoring, the system will dynamically adjust the prompt according to the manual judgment to improve the scoring accuracy for similar answers in the future.

[0079] In the specific process of scoring short-answer questions in finance:

[0080] Obtain the question, and the specific question content is: Please explain what the Capital Asset Pricing Model (CAPM) is and state its formula.

[0081] Obtain the reference answer, and the specific content of the reference answer is:

[0082] <2>The Capital Asset Pricing Model (CAPM) is used to determine the expected return of an asset. <!--2-->

[0083] <3>The CAPM formula is: E(Ri) = Rf + βi * (E(Rm) - Rf), where E(Ri) is the expected return of the asset, Rf is the risk-free rate, βi is the systematic risk coefficient of the asset, and E(Rm) is the expected return of the market. <!--3-->

[0084] Obtain the student's answer content, that is, the short answer question answer information. The specific content is:

[0085] CAPM is a model used to calculate the expected return of an asset. The formula is E(Ri) = Rf + βi * (E(Rm) - Rf).

[0086] The specific scoring process is as follows:

[0087] The model first decomposes the reference answer into two key points.

[0088] For the first key point (<2>< / 2>), the model determines that the student's answer mentions that "CAPM is a model used to calculate the expected return of an asset", so it gives a judgment of "True".

[0089] For the second key point (<3><!--3-->) The model determines that the student's answer mentions that "the formula is E(Ri) = Rf + βi * (E(Rm) - Rf)", so it gives a judgment of "True".

[0090] The model generates feedback, indicating that the student's answer is completely correct and explains the content of each key point.

[0091] Prompt dynamic feedback mechanism:

[0092] If the model's confidence in the judgment of a certain key point is low, the system will request manual intervention.

[0093] After manual scoring, the system will dynamically adjust the prompt according to the manual judgment to improve the scoring accuracy of similar answers in the future.

[0094] In the specific process of applying it to the scoring of artificial intelligence short answer questions:

[0095] Obtain the question. The specific question content is: Please explain what a convolutional neural network (CNN) is and illustrate its application in image processing.

[0096] Obtain the reference answer. The specific content of the reference answer is:

[0097] <2>A convolutional neural network (CNN) is a deep learning model that is particularly suitable for image processing tasks. <!--2-->

[0098] <3>The CNN extracts image features through convolutional layers, and the pooling layer reduces the feature dimension. <!--3-->

[0099] <1>Classification is performed through fully connected layers. <!--1-->

[0100] Obtain the student's answer content, that is, the short answer question answer information. The specific content is:

[0101] CNN is a deep learning model for image processing. It extracts features through convolutional layers, reduces the dimension through pooling layers, and finally performs classification through non-fully connected layers.

[0102] The specific scoring process is as follows:

[0103] The model first decomposes the reference answer into three key points.

[0104] For the first key point (<2><!--2-->) The model determines that the student's answer mentions that "CNN is a deep learning model for image processing", so it gives a judgment of "True".

[0105] For the second key point (<3><!--3-->) The model determines that the student's answer mentions that "the convolutional layer extracts features, the pooling layer reduces the dimension, and the fully connected layer performs classification", so it gives a judgment of "True".

[0106] For the third key point (<1><!--1-->) The model determines that the student's answer mentions "non-fully connected layers", so it gives a judgment of "Fa l se".

[0107] The model generates feedback, indicating that the student's answer is partially correct and explains the content of each key point.

[0108] Prompt dynamic adjustment mechanism:

[0109] If the model's confidence in judging a certain key point is low, the system will request manual intervention.

[0110] After manual scoring, the system will dynamically adjust the prompt according to the manual judgment to improve the scoring accuracy of future similar answers.

[0111] This embodiment can be deployed on NVIDIA A100 GPUs to ensure high-performance real-time processing.

[0112] This embodiment manages API calls through FastAPI and uses vLLM to support asynchronous inference to improve throughput.

[0113] This embodiment integrates pre-trained models through Transformers and Modelscope to ensure the best performance of the model.

[0114] In this embodiment, greedy search is used to ensure the determinism and repeatability of the model output.

[0115] In this embodiment, the Cohen's Kappa index is used to evaluate the consistency between the model scores and human raters, ensuring the reliability of the scores.

[0116] The above described the short-answer question automatic scoring method based on the large language model in the embodiments of the present invention. Next, the short-answer question automatic scoring device based on the large language model in the embodiments of the present invention will be described. Please refer to Figure 3 , the short-answer question automatic scoring device based on the large language model in the embodiments of the present invention includes, for the above embodiments:

[0117] An information acquisition module 301, configured to acquire question information, score information, reference answer information, and short-answer question response information;

[0118] A key point extraction and scoring module 302, configured to extract key delimiters in the reference answer information, decompose the reference answer information into several key points according to the key delimiters, and evenly distribute the scores to each key point according to the score information and the key points;

[0119] A detailed scoring module 303, configured to call the large language model to evaluate the short-answer question response information according to the key points, judge one by one whether the short-answer question response information covers the key point and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback;

[0120] A prompt word dynamic adjustment module 304, configured to generate the confidence of the short-answer question response information, send the short-answer question response information with a confidence lower than the threshold to the manual scoring intervention terminal, obtain the scoring result information sent by the manual intervention terminal, and dynamically adjust the prompt word according to the scoring result information;

[0121] A scoring result sending module 305, configured to send the scoring result information or the detailed evaluation information to the student terminal.

[0122] The information acquisition module includes a question information acquisition unit, a score information acquisition unit, a reference answer information acquisition unit, and a short-answer question response acquisition unit;

[0123] A question information acquisition unit, configured to acquire question information;

[0124] A score information acquisition unit, configured to acquire score information;

[0125] A reference answer information acquisition unit, configured to acquire reference answer information;

[0126] A short-answer question response acquisition unit, configured to acquire short-answer question response information.

[0127] The key point extraction and scoring module includes: a key delimiter extraction unit and a scoring unit;

[0128] The key delimiter extraction unit is used to extract the key delimiters in the reference answer information. The key delimiters include semicolon delimiters and period delimiters; the paragraph before the first key delimiter is taken as a key point, and the paragraphs between every two delimiters are respectively taken as a key point;

[0129] The scoring unit is used to obtain the score information and evenly distribute the score to each key point.

[0130] above Figure 3 The short answer question automatic scoring device based on the large language model in the embodiment of the present invention is described in detail from the perspective of modular functional entities. Next, the electronic device in the embodiment of the present invention will be described in detail from the perspective of hardware processing.

[0131] Figure 4 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 400 may vary greatly due to different configurations or performances, and may include one or more processors 410 (for example, one or more processors) and a memory 420, and one or more storage media 430 for storing application programs 433 or data 432 (for example, one or more storage devices, including RAM, FLASH, etc.). Among them, the memory 420 and the storage media 430 may be transient storage or persistent storage. The program stored in the storage media 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the electronic device 400. Further, the processor 410 may be configured to communicate with the storage media 430 and execute a series of instruction operations in the storage media 430 on the electronic device 400.

[0132] The electronic device 400 may further include one or more power supplies 440, one or more input / output interfaces 450, and / or one or more operating systems 431, such as FreeRTOS, Android, etc. Those skilled in the art can understand that Figure 4 The shown structure of the electronic device does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0133] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the method for automatically grading short-answer questions based on a large language model.

[0134] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, or unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a mobile device, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0136] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for automatically grading short-answer questions based on large language models, characterized in that, The short-answer question automatic scoring method based on a large language model includes: Obtain question information, score information, reference answer information, and short-answer question response information; Extract the key delimiters in the reference answer information, decompose the reference answer information into several key points according to the key delimiters, and evenly distribute the scores to each key point according to the score information and the key points; Call the large language model to evaluate the short-answer question response information according to the key points, judge one by one whether the short-answer question response information covers the key point according to the key points and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback; Generate the confidence level of the short-answer question response information, send the short-answer question response information with a confidence level lower than the threshold to the manual scoring intervention end, obtain the scoring result information sent by the manual intervention end, and dynamically adjust the prompt words according to the scoring result information; Send the scoring result information or the detailed evaluation information to the student end.

2. The short answer question automatic scoring method based on a large language model according to claim 1, wherein Before generating detailed evaluation information for each binary feedback, it also includes: Filter non-academic language through a keyword filter, and the non-academic language includes emotional appeal language and role-playing language.

3. The short answer question automatic scoring method based on a large language model according to claim 2, wherein The extracting the key delimiters in the reference answer information, decomposing the reference answer information into several key points according to the key delimiters, and evenly distributing the scores to each key point according to the score information and the key points includes: Extract the key delimiters in the reference answer information, and the key delimiters include semicolon delimiters and period delimiters; Take the paragraph before the first key delimiter as a key point, and take the paragraphs between every two delimiters as a key point respectively; Obtain the score information and evenly distribute the scores to each key point.

4. The short answer question automatic scoring method based on a large language model according to claim 3, wherein The calling the large language model to evaluate the short-answer question response information according to the key points, judging one by one whether the short-answer question response information covers the key point according to the key points and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback includes: Call the Llama-3.1-70B large language model to evaluate the short-answer question information; Judge one by one whether the short-answer question response information covers the key point according to the key points. If it covers, generate the binary feedback as a True feedback. If it does not cover, generate the binary feedback as a False feedback; Generate detailed evaluation information for each binary feedback, and at the same time assign scores to the short-answer question response information.

5. The short answer question automatic scoring method based on a large language model according to claim 4, wherein The sending the scoring result information or the detailed evaluation information to the student end includes: Judge whether the confidence level is lower than the threshold. If so, send the scoring result information to the student end. If not, send the detailed evaluation information to the student end.

6. A short-answer question automatic scoring device based on a large language model, for the short-answer question automatic scoring method based on a large language model as described in claim 5, characterized in that, It includes: An information acquisition module for obtaining question information, score information, reference answer information, and short-answer question response information; A key point extraction and scoring module for extracting the key delimiters in the reference answer information, decomposing the reference answer information into several key points according to the key delimiters, and evenly distributing the scores to each key point according to the score information and the key points; A detailed scoring module, which is used to call a large language model to evaluate the short answer question response information based on key points, judge whether the short answer question response information covers each key point one by one and generate corresponding binary feedback, and generate detailed evaluation information for each binary feedback; A prompt dynamic adjustment module, which is used to generate the confidence level of the short answer question response information, send the short answer question response information with a confidence level lower than the threshold to the manual scoring intervention terminal, obtain the scoring result information sent by the manual intervention terminal, and dynamically adjust the prompt according to the scoring result information; A scoring result sending module, which is used to send the scoring result information or the detailed evaluation information to the student terminal.

7. The short answer question automatic scoring device based on the large language model according to claim 6, characterized in that, The information acquisition module includes a question information acquisition unit, a score information acquisition unit, a reference answer information acquisition unit, and a short answer question response acquisition unit; The question information acquisition unit is used to acquire question information; The score information acquisition unit is used to acquire score information; The reference answer information acquisition unit is used to acquire reference answer information; The short answer question response acquisition unit is used to acquire short answer question response information.

8. The short answer question automatic scoring device based on a large language model according to claim 7, characterized in that, The key point extraction and scoring module includes: a key separator extraction unit and a scoring unit; The key separator extraction unit is used to extract the key separators in the reference answer information, and the key separators include semicolon separators and period separators; regard the paragraph before the first key separator as a key point, and regard each paragraph between every two separators as a key point respectively; The scoring unit is used to obtain the score information and evenly distribute the score to each key point.

9. An electronic device, the electronic device includes a memory and at least one processor, and instructions and data are stored in the memory; The at least one processor calls the instructions and data in the memory so that the electronic device executes each step of the automatic short answer question scoring method based on a large language model as described in any one of claims 1-5.

10. A readable storage medium, on which instructions and data are stored, characterized in that, When the instructions are executed by the processor, each step of the automatic short answer question scoring method based on a large language model as described in any one of claims 1-5 is implemented.

Images