An online experiment report automatic marking method and system for electronic information communication major

Abstract

The application provides an electronic information communication professional online experiment report automatic marking method and system, and belongs to the technical field of teaching information processing. The method comprises the following steps: obtaining target answering information and a target title type; if the target title type is a first title type, comparing the target answering information with a first standard answer to generate a first score; if the target title type is a second title type, generating a second score by calculating the similarity between the target answering information and a second standard answer; if the target title type is a third title type, marking the target answering information by a target large language model and outputting a third score; and if the target title type is a fourth title type, extracting feature information from the target answering information, generating corresponding circuit system code according to the feature information, inputting the circuit system code into a target mixed signal modeling language model, outputting a fourth score and generating a marking result. The application can realize automatic marking of electronic information communication professional online experiment reports.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of teaching information processing, and particularly relates to an online experimental report automatic marking method and system for electronic information communication major. BACKGROUND

[0002] Automatic marking is a technology based on natural language processing, which is usually used to mark a large number of teaching documents such as homework and examination papers, etc., so as to speed up the marking speed and improve the marking efficiency. However, the existing automatic marking technology can only be used to mark the conventional question types, while the electronic online experimental report of electronic communication major involves a variety of complex question types, especially the short answer questions and circuit design questions, and the existing automatic marking method cannot meet the marking requirements of the online experimental report of electronic communication major. SUMMARY

[0003] Therefore, the present application aims to overcome the deficiencies in the prior art, and provides an online experimental report automatic marking method and system for electronic information communication major, so as to realize the automatic marking of the online experimental report of electronic communication major.

[0004] In a first aspect, the present application provides an online experimental report automatic marking method for electronic information communication major, comprising:

[0005] obtaining the answer information of the online experimental report, determining the target answer information to be marked and the target question type corresponding to the target answer information from the answer information;

[0006] if the target question type is a first question type, comparing the target answer information with a first standard answer to obtain a comparison result, and generating a first score according to the comparison result; the first question type includes a selection question, a judgment question and a numerical fill-in-the-blank question;

[0007] if the target question type is a second question type, calculating the similarity between the target answer information and a second standard answer through a similarity calculation formula, and generating a second score according to the similarity; the second question type includes a character fill-in-the-blank question;

[0008] if the target question type is a third question type, constructing an initial large language model, adjusting the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes a short answer question, inputting the target answer information into the target large language model, and outputting a third score;

[0009] If the target question type is a fourth question type, extracting feature information from the target answer information, generating a corresponding circuit system code based on the feature information, inputting the circuit system code into a target mixed-signal modeling language model, and outputting a fourth score; the fourth question type includes a circuit design question;

[0010] An automatic review result is generated according to the first score, the second score, the third score, and the fourth score.

[0011] In one embodiment, calculating the similarity between the target answer information and the second standard answer using a similarity calculation formula includes:

[0012] Calculating the cosine similarity and Jaccard similarity between the target answer information and the second standard answer respectively;

[0013] The similarity between the target answer information and the second standard answer is calculated according to the following similarity calculation formula:

[0014] T = w × Sim + (1-w) × Jaccard;

[0015] Among them, T is similarity, Sim is cosine similarity, Jaccard is Jaccard similarity, w is weight, 0<=w<=1, w=1-e -β*length , length is the length of the reference answer, and β is a positive coefficient.

[0016] In one embodiment, adjusting the initial large language model by using a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model includes:

[0017] Obtaining weight parameters of the initial large language model;

[0018] Freezing the weight parameters of the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain an intermediate large language model;

[0019] An enhanced training dataset is obtained using a hybrid interactive data augmentation method, and the intermediate large language model is trained using the enhanced dataset and a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain the target large language model.

[0020] In one embodiment, the training process of the target large language model includes:

[0021] Obtaining short-answer questions and standard answers corresponding to the short-answer questions;

[0022] Inputting the short-answer question and the standard answer to the short-answer question into the answer generation module of the initial large language model, and causing the answer generation module to output a plurality of derived answers corresponding to the standard answer to the short-answer question;

[0023] Inputting the short-answer question and the corresponding multiple derived answers into the automatic review module of the initial large language model, and allowing the automatic review module to output review results and scores corresponding to the derived answers to the short-answer question;

[0024] Correcting the derived answer to the short-answer question and the corresponding review result and score to obtain a corrected derived answer to the short-answer question and the corresponding corrected review result and score;

[0025] Generating multiple training data sets through instruction fine-tuning according to the short-answer question titles, the standard answers to the short-answer questions, the multiple revised derived answers, and the revised review results and scores;

[0026] Inputting each of the training data sets into a secondary decomposition low-rank adaptive fine-tuning module for model training;

[0027] By formula:

[0028] W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =W (0) +B CUR A CUR

[0029] The SVD decomposition and CUR decomposition are combined into a secondary low-rank decomposition, and the secondary low-rank decomposition process is embedded into the loss function of the secondary decomposition low-rank adaptive fine-tuning module; wherein W (0) It is the weight parameter matrix of the large language model pre-training, which is the part that is frozen and does not participate in fine-tuning training. ΔW=BA, B∈R m×r , A∈R r×n are two low-rank matrices, r is the low-rank dimension, m and n are the dimensions of the weight matrix;

[0030] The low-rank regularization function is defined by the training algorithm:

[0031]

[0032] Represents the low-rank approximation of the matrix, CU cur R represents the low-rank approximate matrix obtained by performing SVD and CUR secondary decomposition on BA, represents the Frobenius norm;

[0033] And through the formula:

[0034] L total =L task +λ SVD-CUR R SVD-CUR (B,A)

[0035] The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation; L task is the loss function in the original LoRA fine-tuned model;

[0036] The low-rank matrices y and y are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence criterion, it is determined that the training of the target large language model is completed.

[0037] In one embodiment, the target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model, and a key function performance indicator evaluation model, and the method includes:

[0038] The answer to the question is converted into a circuit system code described in a mixed signal modeling language by the circuit description automatic generator;

[0039] Review and score the circuit code using the code review language model, and output a static score;

[0040] Through the key functional performance indicator evaluation model, various key functional performance indicators of the circuit code are constructed, and the circuit code is run to measure the indicators and output dynamic scores.

[0041] In one embodiment, converting the answer to the question into a circuit system code described in a mixed signal modeling language by the automatic circuit description generator includes:

[0042] Obtaining circuit information from the answer corresponding to the circuit design question; the circuit information includes components and algorithm modules;

[0043] Calling the factory mode of the circuit description automatic generator, generating corresponding instantiation code described in a mixed-signal modeling language for each component and algorithm module based on the standard algorithm modules and standard components already in the database, and obtaining instantiated components and instantiated algorithm modules;

[0044] Defining instantiated component ports or instantiated algorithm module ports through the circuit description automatic generator, and configuring port parameters;

[0045] According to the port parameters, the connection relationship of each instantiated component or instantiated algorithm module is determined to obtain the circuit system code described in the target mixed-signal modeling language.

[0046] In one embodiment, inputting the circuit system code into a target mixed-signal modeling language model and outputting a fourth score includes:

[0047] Inputting the circuit system code into a circuit system code review language model in a target mixed-signal modeling language model to perform circuit static review, and outputting a circuit system static score corresponding to the circuit system code;

[0048] Use the key functional performance indicator evaluation model to predefine the target indicators of the circuit system code, and build a target indicator calculation module in the target mixed-signal modeling language model;

[0049] Using the key functional performance index evaluation model, a mixed signal modeling language simulator is used to load the circuit system code described in the mixed signal modeling language converted from the actual answer to the circuit design problem, and at the same time load the target index calculation module preset for the circuit design problem;

[0050] Running the circuit system code corresponding to the actual answer to the circuit design question using the key functional performance indicator evaluation model to output a target indicator value, inputting the target indicator value into a dynamic review module of the target key functional performance indicator evaluation model to output a circuit system dynamic score corresponding to the circuit system code;

[0051] The circuit system static score and the circuit system dynamic score are weighted to obtain the fourth score.

[0052] In one embodiment, determining target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information includes:

[0053] Dividing the answer information of the online experiment report into multiple answer areas, each answer area corresponding to a question type;

[0054] Determine a target answer area corresponding to the target answer information to be reviewed, and determine a target question type corresponding to the target answer information from the question types based on the target answer area.

[0055] In one embodiment, if the target question type is the first question type, comparing the target answer information with the first standard answer to obtain a comparison result includes:

[0056] If the target question type is a multiple-choice question or a true-or-false question, the target answer information is compared with the standard answer corresponding to the multiple-choice question or the fill-in-the-blank question to determine whether the comparison is successful or failed;

[0057] If the target question type is a numerical fill-in-the-blank question, determining a numerical range corresponding to a standard answer to the numerical fill-in-the-blank question;

[0058] The target answer information is compared with the numerical range to determine whether the target answer information is within the numerical range.

[0059] In a second aspect, the present application provides an automatic review system for online experimental reports of electronic information and communications majors, including:

[0060] An acquisition module is used to obtain answer information of the online experiment report, and determine the target answer information to be reviewed and the target question type corresponding to the target answer information from the answer information;

[0061] a first scoring module configured to compare the target answer information with a first standard answer if the target question type is a first question type, obtain a comparison result, and generate a first score based on the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions;

[0062] a second scoring module configured to calculate the similarity between the target answer information and the second standard answer using a similarity calculation formula if the target question type is a second question type, and generate a second score based on the similarity; the second question type includes a text-based fill-in-the-blank question;

[0063] A third scoring module is configured to, if the target question type is a third question type, construct an initial large language model, adjust the initial large language model using a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes short-answer questions, input the target answer information into the target large language model, and output a third score;

[0064] a fourth scoring module configured to, if the target question type is a fourth question type, extract feature information from the target answer information, generate a corresponding circuit system code based on the feature information, input the circuit system code into a target mixed-signal modeling language model, and output a fourth score; the fourth question type includes a circuit design question;

[0065] A result generating module is used to generate an automatic review result according to the first score, the second score, the third score and the fourth score.

[0066] The embodiments of the present invention have the following beneficial effects:

[0067] The present invention provides a method for automatically marking online experimental reports of electronic information and communication majors. First, the corresponding target question type is determined through the answer information of the experimental report, and corresponding marking rules are set according to different question types. This realizes the automatic marking of online experimental reports of electronic information and communication majors, and improves the reliability and scientificity of the marking results. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0069] Figure 1 A flowchart of a method for automatically reviewing online experimental reports for electronic information and communications professionals provided in an embodiment of the present application;

[0070] Figure 2 A flowchart of a first question type review method provided in an embodiment of the present application;

[0071] Figure 3 A flowchart of a target large language model training method provided in an embodiment of the present application;

[0072] Figure 4 A schematic diagram of the framework structure of an automatic review system for online experimental reports of electronic information and communications majors provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0074] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the template description herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0076] See also Figure 1 , Figure 1A flowchart of a method for automatically reviewing online experiment reports for electronic information and communications majors provided in an embodiment of the present application is provided. The method can be used to automatically review online experiment reports for electronic information and communications majors. The method includes steps S101-S106.

[0077] S101, obtaining answer information of an online experiment report, and determining target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information.

[0078] Online experiments in the electronics and communications major are electronic reports generated by students using electronics and communications hardware experiment boxes or virtual simulation software. These reports are generally divided into five types of questions: multiple-choice, true-or-false, fill-in-the-blank, short-answer, and circuit design. Students fill in the corresponding information for each question type, and their responses are then graded and reviewed using various methods.

[0079] To facilitate grading, the answers in online lab reports are usually divided into multiple answer areas in advance, each corresponding to a question type. During grading, simply determine the target answer area for the target answer information to be graded. Then, based on the target answer area, determine the target question type corresponding to the target answer information from the question types. Finally, determine the grading strategy for the target question type based on the target question type to achieve automatic grading.

[0080] S102: If the target question type is the first question type, the target answer information is compared with the first standard answer to obtain a comparison result, and a first score is generated according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions.

[0081] See also Figure 2 , step S102 includes steps S1021-S1023.

[0082] S1021: If the target question type is a multiple-choice question or a true-or-false question, the target answer information is compared with the standard answer corresponding to the multiple-choice question or the fill-in-the-blank question to determine whether the comparison is successful or failed.

[0083] S1022: If the target question type is a numerical fill-in-the-blank question, determine the numerical range corresponding to the standard answer of the numerical fill-in-the-blank question.

[0084] S1023: Compare the target answer information with the numerical range to determine whether the target answer information is within the numerical range.

[0085] The answer to the multiple-choice question or the judgment question is generally fixed, and therefore, only the target answer information filled in by the student needs to be compared with the standard answer. If the target answer information is the same as the standard answer, it is determined that the comparison is successful, which indicates that the student fills in the correct answer. Otherwise, it indicates that the student fills in the wrong answer.

[0086] For the numerical fill-in-the-blank question, the standard answer is usually a numerical value or a numerical interval. If it is a numerical value, all the expression forms of the standard answer can be listed as the standard answer. For example, the answer is 0.75, and various forms of the answer such as 0.75, 3 / 4, 75%, etc. can be used as the standard answer, which is compared with the target answer information filled in by the student. If the comparison is successful, it indicates that the answer filled in by the student is the correct answer, and the score of the question can be recorded. If the answer is wrong, no score is recorded. Similarly, if the standard answer is a numerical interval, the same method can be used to set the standard answer, which is compared with the target answer information filled in by the student, and then the score is recorded.

[0087] In S103, if the target question type is the second question type, the similarity between the target answer information and the second standard answer is calculated by using a similarity calculation formula, and a second score is generated according to the similarity. The second question type includes a character fill-in-the-blank question.

[0088] The standard answer of the character fill-in-the-blank question is usually a word, a phrase or a short sentence. For the answer of the character fill-in-the-blank question being a word or a phrase, the Jaccard similarity can be used to compare the similarity between the answer filled in by the student and the standard answer. For the answer being a short sentence or a sentence, the cosine similarity can be used to convert the text into a vector to calculate the semantic similarity of the text, which is particularly suitable when the text uses complex expressions or synonym replacement.

[0089] In order to improve the accuracy of the character fill-in-the-blank question, the cosine similarity and the Jaccard similarity between the target answer information and the second standard answer are calculated respectively.

[0090] Then, the similarity between the target answer information and the second standard answer is calculated according to the following similarity calculation formula:

[0091] T=w×Sim+(1-w)×Jaccard;

[0092] wherein, T is the similarity, Sim is the cosine similarity, Jaccard is the Jaccard similarity, w is the weight, 0<=w<=1, w=1-e -β*length, length is the length of the reference answer, and β is a positive coefficient used to adjust the sensitivity of w to the length of the reference answer. The value of β can be adjusted according to specific scoring requirements. The larger the value, the faster w increases with length.

[0093] In this way, when length is very small, such as 1 or close to 0. If it is close to 1, then w is close to 0, and in this case, Jaccard similarity can be given a higher weight. As length increases, w also gradually increases, and in this case, cosine similarity can be given a higher weight. Finally, a threshold t is set. When the answer similarity T>t, the answer is considered correct.

[0094] S104. If the target question type is the third question type, an initial large language model is constructed, and the initial large language model is adjusted through a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes short-answer questions, and the target answer information is input into the target large language model to output a third score.

[0095] Large language models (LLMs) are deep learning models trained using large amounts of text data. They can generate natural language text or understand the meaning of text. Common large language models, such as ChatGPT-4 and Cluade3, are pre-trained using general knowledge text data. Currently, there are no large language models trained specifically for the electronic communications field. Therefore, in order for large language models to more accurately analyze the accuracy or score of student answers compared to standard answers, it is necessary to fine-tune the pre-trained large language models using professional knowledge from the electronic communications field and a real test bank.

[0096] For short-answer questions, the titles and standard answers are relatively easy to obtain, but diverse student answer data is indeed difficult to obtain. In order to achieve the accuracy and scientificity of the automatic grading results of the target large language model, the student answer data needs to be processed, such as data augmentation.

[0097] Data enhancement processing includes two methods:

[0098] Method 1: First, collect short-answer questions and standard answers from the question bank. For each question, collect as many real student answers and corresponding teacher comments and scores as possible. This real student answers and teacher comments can be collected before the system goes live, or accumulated gradually as the system goes live to serve as training data to continuously optimize the grading accuracy of the large language model. The short-answer questions, standard answers, and real student answers are fed into the pre-trained large language model. The large language model then rewrites the student answers based on the questions and standard answers to generate new student answers.

[0099] To ensure the diversity of generated answers, the model's parameters for generating answers need to be set. This includes specifying the length, complexity, relevance, and correctness of the answers in the instruction prompt. For example, the length of the generated answers can be set to no more than a certain number of words to ensure the conciseness of the answers. At the same time, the creativity and diversity of the answers can be controlled by adjusting the model's temperature parameter. The generated answers, along with the questions and reference answers, are then input into the large language model, which automatically reviews and scores the newly generated answers. Human experts then revise the large language model's review results and scores. This results in a new answer and review result for the question.

[0100] Method 2: Instead of feeding the large language model any actual student answers, the short-answer questions and standard answers from the question bank are directly fed into the large language model. The large language model then generates new answers based on the questions and standard answers. This process also requires setting the model's answer generation parameters in the same way as method 1. The generated answers are then fed back into the large language model along with the questions and standard answers, allowing it to automatically review and score the answers. Human experts then revise the large language model's review and scores. This results in a new answer and review result for the question.

[0101] See also Figure 3 , step S104 also includes steps S1041-S1043.

[0102] S1041. Obtain weight parameters of the initial large language model.

[0103] In order to maximize the accuracy of large language model fine-tuning and reduce the fine-tuning cost, this application proposes a new quadratic decomposition low-rank adaptive fine-tuning algorithm. However, this algorithm requires certain modifications to the source code and parameters of the large language model. Therefore, an open source pre-trained large language model will be used as the initial large language model, such as Llama 3-70B.

[0104] Because open-source pre-trained large language models have a large number of weight parameters, adding new knowledge to the large language model requires adjusting the model's weight parameters through training. Adjusting the weight parameters of the entire pre-trained model is a very costly and time-consuming task. Therefore, it is necessary to freeze the pre-trained weight parameters and only train the newly added knowledge dataset, merging the weight parameters trained with the newly added dataset into the original weight parameters. The currently commonly used method is Low-Rank Adaptation (LoRA).

[0105] S1042: Freeze the weight parameters of the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain an intermediate large language model.

[0106] In order to further improve the training accuracy of LoRA on the newly added data set, reduce the consumption of computing resources and video memory, and save training time, this embodiment proposes a quadratic low-rank decomposition method and fine-tuning algorithm.

[0107] The core idea of ​​the LoRA method is to approximate the weight matrix through low-rank decomposition, thereby reducing the number of parameters and computation. LoRA fine-tunes the model on downstream tasks by adding a low-rank matrix to the side of the original feature. This address matrix is ​​implemented by the product of two small matrices, expressed as:

[0108] W=W (0) +ΔW=BA.

[0109] Where W (0) is the weight parameter matrix of the large language model pre-training, which is the part that is frozen and does not participate in fine-tuning training, and ΔW = BA, where B∈R m×r , A∈R r×n Are two low-rank matrices, which are low-rank dimensions, usually much smaller than the dimensions m and n of the weight matrix.

[0110] And through the formula:

[0111] L total =L task +λ SvD-CUR R SvD-CUR (B,A)

[0112] The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation; L task is the loss function in the original LoRA fine-tuned model;

[0113] The low-rank matrices y and y are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence criterion, it is determined that the training of the target large language model is completed.

[0114] The weight matrix is ​​decomposed using singular value decomposition (SVD), which is a classic low-rank decomposition method. By adjusting the number of retained singular values, the rank of the low-rank matrix can be controlled.

[0115] First, perform SVD decomposition on the original weight matrix ΔW:

[0116] ΔW=UΣV T

[0117] Select the first singular value and its corresponding singular vector:

[0118]

[0119] Among them U k ∈R m×k ,Σ k ∈R k×k , V k ∈R n×k By setting the k value, the rank of the low-rank matrix can be dynamically controlled, and different ranks can be used for different modules in the model.

[0120] CUR decomposition is a decomposition method based on matrix columns and rows, which can better preserve the structural information of the original matrix.

[0121] Based on the preliminary SVD decomposition, CUR decomposition is performed:

[0122] Select the key columns c and rows r. There are many ways to select key columns and rows. To keep the formula simple, we will directly select the first c columns C and the first r rows R:

[0123] C=ΔW SVD [:,top-c columns]

[0124] R=ΔW SVD [top-r rows,:]

[0125] Calculate the intermediate matrix U cur :U cur =(C T C) -1 C T ΔW SVD R T (W T ) -1

[0126] The final CUR low-rank approximation matrix is:

[0127] ΔW CUR ≈CU cur R

[0128] where C∈R m×c , R∈R r×n , U∈R c×r .

[0129] The above low-rank approximation matrix is ​​expressed as the product of two matrices so that it can be applied to the LoRA formula.

[0130] Define A and B as follows:

[0131] A CUR =U cur R

[0132] B CUR =C

[0133] therefore

[0134] ΔW CUR ≈B CUR A CUR

[0135] This gives the form required in the LoRA formula:

[0136] W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =W (0) +B CUR A CUR

[0137] S1043. Use a hybrid interactive data enhancement method to obtain an enhanced training dataset, and train the intermediate large language model using the enhanced dataset and a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain the target large language model.

[0138] The training process of the target large language model includes:

[0139] Obtaining short-answer questions and standard answers corresponding to the short-answer questions;

[0140] Inputting the short-answer question and the standard answer to the short-answer question into the answer generation module of the initial large language model, and causing the answer generation module to output a plurality of derived answers corresponding to the standard answer to the short-answer question;

[0141] Inputting the short-answer question and the corresponding multiple derived answers into the automatic review module of the initial large language model, and allowing the automatic review module to output review results and scores corresponding to the derived answers to the short-answer question;

[0142] Correcting the derived answer to the short-answer question and the corresponding review result and score to obtain a corrected derived answer to the short-answer question and the corresponding corrected review result and score;

[0143] Based on the short-answer questions, the standard answers to the short-answer questions, the multiple revised derivative answers, and the revised review results and scores, multiple training data sets are generated through instruction fine-tuning.

[0144] Then, each of the training data sets is input into a secondary decomposition low-rank adaptive fine-tuning module for model training;

[0145] By formula:

[0146] W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =W (0) +B CUR A CUR

[0147] Combining singular value decomposition and CUR decomposition into a quadratic low-rank decomposition, and embedding the quadratic low-rank decomposition process into the loss function of the quadratic decomposition low-rank adaptive fine-tuning module;

[0148] The low-rank regularization function is defined by the training algorithm:

[0149]

[0150] To represent the low-rank approximation of the matrix, CU cur R represents the low-rank approximate matrix obtained by performing SVD and CUR secondary decomposition on BA, represents the Frobenius norm.

[0151] And through the formula:

[0152] L total =L task +λ SVD-CUR R SVD-CUR (B,A)

[0153] The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation;

[0154] The low-rank matrices B and A are optimized using gradient descent. After multiple iterations, if the loss function reaches convergence, the training of the target large language model is considered complete. By inputting the answers to the short-answer questions in the online lab report into the target large language model, the corresponding third score is automatically generated.

[0155] S105. If the target question type is the fourth question type, extract feature information from the target answer information, generate a corresponding circuit system code based on the feature information, input the circuit system code into the target mixed signal modeling language model, and output a fourth score; the fourth question type includes a circuit design question.

[0156] The target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model, and a key functional performance indicator evaluation model.

[0157] The automatic circuit description generator is responsible for converting the answers to the questions into circuit code described in the Verilog-AMS mixed-signal modeling language; the code review language model is responsible for reviewing and scoring the circuit code and outputting a static score; and the key functional performance indicator evaluation model is responsible for constructing various key functional performance indicators for measuring the circuit code, running the circuit code to measure the indicators, and outputting a dynamic score.

[0158] Verilog-AMS is a comprehensive mixed-signal modeling language that combines Verilog-HDL (for the digital part) and Verilog-A (for the analog part), allowing complex digital, analog and mixed-signal systems to be described in the same model.

[0159] Since the component library used by the respondent is known, the corresponding component code library will be written in the circuit description automatic generator in advance, and the algorithm module used by the respondent is also known, and the corresponding algorithm module code will be written in the circuit description automatic generator in advance. If it is a module developed by the respondent, it is also written in the Verilog-AMS development language by default and used directly in the circuit description automatic generator. Then the automatically generated Verilog-AMS description of the corresponding circuit is only to select the components and module codes in the automatic generator and connect the ports. The automatic generation of Verilog-AMS circuit code will mainly focus on converting these specific information into corresponding Verilog-AMS code.

[0160] Therefore, the review of circuit design drawings can be converted into the review of the Verilog-AMS codes corresponding to the circuit design drawings.

[0161] Similar to short-answer questions, to enable the large language model to more accurately analyze the accuracy of students' answers and their scores, it is necessary to fine-tune the pre-trained large language model using specialized knowledge from the electronics and communications field and a real-world question bank. Furthermore, there is no single, standard answer for design questions; multiple circuit construction methods may be correct for the same question, making grading more flexible and varied. This requires carefully constructed training datasets to fine-tune the large language model.

[0162] First, it is necessary to perform data augmentation on the answer information corresponding to the circuit design questions. The data augmentation method is the same as that for the short-answer questions. Then, the model is trained using the data-augmented answer information to improve the grading accuracy of the grading model corresponding to the circuit design questions.

[0163] In one embodiment, extracting characteristic information from the target answer information and generating a corresponding circuit system code according to the characteristic information includes:

[0164] Obtaining target answer information corresponding to the circuit design question, and extracting the characteristic information from the target answer information; the characteristic information includes the type, quantity, port, topology, and parameters of electronic components and algorithm modules;

[0165] The characteristic information is converted into the corresponding circuit system code through a circuit description automatic generator.

[0166] First, the circuit information collected by the test-taker, either from the experiment box or simulation software, must be analyzed to determine the model, quantity, and connection information of the components and modules. At the same time, the consistency and completeness of this information must be verified to ensure there are no omissions or errors.

[0167] Based on the module and component definitions in the existing library, the corresponding instantiation code is generated for each component. This includes defining an instance of each module and passing the correct parameters to the module.

[0168] Instantiating prefabricated Verilog-AMS components and algorithm modules is an important step in taking the design from the module level to the specific implementation. This involves converting the abstract module description into concrete hardware description language code. The process mainly includes:

[0169] Step 1: Define the module interface

[0170] Each module and component has a defined interface, including input ports, output ports, and necessary control signals. This is given in the module definition file, such as the module declaration in Verilog-AMS.

[0171] Step 2: Get module parameters

[0172] A module may require specific parameters to configure its behavior, such as the bit width of the counter, the size of the memory, etc. These parameters need to be explicitly specified during instantiation.

[0173] Step 3: Automatically generate instantiation code

[0174] Use the factory pattern to instantiate each corresponding prefabricated component or module using the hardware description language based on the components and algorithm modules used in the circuit built by the test-taker. The instantiation process includes specifying the component or module name, the connected ports, and the parameters to be passed.

[0175] Step 4: Parameter and port mapping

[0176] Map the actually used signals, data lines, and control lines to the ports defined by the module. This involves matching port names and converting data (such as bit width adjustment) when necessary.

[0177] Through such steps, any pre-prepared Verilog-AMS component and algorithm module can be instantiated into a larger Verilog-AMS circuit design to form a complete system-level circuit design.

[0178] Then, according to the port connection information provided in the answer information, the code for connecting the modules inside the Verilog-AMS described circuit is generated. This involves the definition and assignment of signal lines between instances, and needs to ensure that different modules can interact correctly and transmit data and control signals. This process can be divided into several specific steps, each of which is part of the automatic code generation.

[0179] The working principle of the circuit description automatic generator of the target mixed signal modeling language model for generating the code of the connection between modules includes the following steps:

[0180] Step 1: Analyze the port connection information

[0181] First, parse the connection relationship between modules from the provided information. This is usually a pair consisting of the port name of a module and the target port name to which it is connected. For example, a JSON file or other format data can be used to describe which port of which module is connected to which port of another module.

[0182] Step 2: Define signal lines

[0183] According to the parsed connection information, define the signal lines that need to be implemented in the Verilog-AMS design to realize these connections. This includes generating a unique signal name for each connection and specifying the width and type of the signal (e.g., single-bit signal, multi-bit vector, clock signal, etc.).

[0184] Step 3: Generate connection statements

[0185] Generate Verilog-AMS statements for each connection, which specify the output port of one module connected to the input port of another module.

[0186] By converting the answer information in the circuit design diagram into code described by the mixed signal modeling language, and then inputting the code into the review module of the target circuit system code review large language model, the automatic review of the circuit design diagram can be realized.

[0187] In one embodiment, the inputting the circuit system code into the target mixed signal modeling language model and outputting the fourth score comprises:

[0188] Inputting the circuit system code into the circuit system code review large language model in the target mixed signal modeling language model for circuit static review, and outputting the circuit system static score corresponding to the circuit system code;

[0189] Using the key functional performance indicator evaluation model in the target mixed-signal modeling language model, predefine the target indicators of the circuit system code, and build a target indicator calculation module in the target mixed-signal modeling language model;

[0190] Using the key functional performance index evaluation model, a mixed signal modeling language simulator is used to load the circuit system code described in the mixed signal modeling language converted from the actual answer to the circuit design problem, and at the same time load the target index calculation module preset for the circuit design problem;

[0191] Running the circuit system code corresponding to the actual answer to the circuit design question using the key functional performance indicator evaluation model to output a target indicator value, inputting the target indicator value into a dynamic review module of the target key functional performance indicator evaluation model to output a circuit system dynamic score corresponding to the circuit system code;

[0192] The circuit system static score and the circuit system dynamic score are weighted to obtain the fourth score.

[0193] The specific calculation method is as follows:

[0194] The circuit designed and constructed by the examinee is reviewed and scored based on the code review language model within the target circuit system. This is a static evaluation of the circuit code described in Verilog-AMS. In addition to static evaluation of the circuit's design logic, a dynamic evaluation of various functional performance indicators during circuit operation is also required using the key functional performance evaluation model within the target mixed-signal modeling language model. Therefore, a comprehensive evaluation of a circuit requires a combination of static and dynamic evaluations.

[0195] Among them, the fourth score = W1×large model score + W2×key function performance index 1 score + W3×key function performance index 2 score + ... + Wn×key function performance index n-1 score + Wn×key part topology score, where W1 to Wn are weight parameters that can be customized by the question setter, W1+...+Wn=1,0 <Wn<1。其中W1是静态评分的权重,而W2……Wn动态评分中各个关键功能性能指标的权重。

[0196] As for the performance indicators of the circuit, different performance indicators can be determined for different circuits.

[0197] For example, for digital circuits, signal propagation delay, time for a signal to change from low to high (or high to low), power consumption, and noise capacity can be used as main performance indicators. For analog circuits, the ratio of the output signal of an amplifier to the input signal, the response of the circuit to different frequency signals, the signal-to-noise ratio, and the proportion of harmonic components in the output signal can be used as main performance indicators. Users can also select appropriate performance indicators according to actual conditions.

[0198] S106, generating an automatic marking result according to the first score, the second score, the third score and the fourth score.

[0199] According to the first score, the second score, the third score and the fourth score, the comprehensive score of each type of question is calculated, and the comprehensive score is taken as the automatic marking result, so that the automatic marking of the experiment report is realized.

[0200] The electronic information communication professional online experiment report automatic marking method provided in the embodiment first determines the corresponding question type through the answer information of the experiment report, and sets corresponding marking rules according to different question types, so as to realize the automatic marking of the electronic information communication professional online experiment report, and improve the reliability and scientificity of the marking result.

[0201] Referring to Figure 4 The embodiment of the present application also provides an electronic information communication professional online experiment report automatic marking system 400, which comprises:

[0202] The acquisition module 401 is configured to acquire answer information of an online experiment report, determine target answer information to be marked and a target question type corresponding to the target answer information from the answer information;

[0203] The first scoring module 402 is configured to compare the target answer information with a first standard answer if the target question type is a first question type, obtain a comparison result, and generate a first score according to the comparison result; the first question type includes a selection question, a judgment question and a numerical fill-in-the-blank question;

[0204] The second scoring module 403 is configured to calculate the similarity between the target answer information and a second standard answer by using a similarity calculation formula if the target question type is a second question type, and generate a second score according to the similarity; the second question type includes a character fill-in-the-blank question;

[0205] The third scoring module 404 is configured to construct an initial large language model if the target question type is a third question type, adjust the initial large language model by using a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes a short answer question, input the target answer information into the target large language model, and output a third score.

[0206] a fourth scoring module 405 configured to, if the target question type is a fourth question type, extract feature information from the target answer information, generate a corresponding circuit system code based on the feature information, input the circuit system code into a target mixed-signal modeling language model, and output a fourth score; the fourth question type includes a circuit design question;

[0207] The result generating module 406 is configured to generate an automatic review result according to the first score, the second score, the third score, and the fourth score.

[0208] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not limiting, and thus other examples of the exemplary embodiments may have different values.

[0209] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0210] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that variations and modifications are possible without departing from the scope of the present invention, and such variations and modifications are fully within the scope of protection of the present invention.

Claims

1. A method for automatically reviewing online experimental reports for electronic information and communication majors, characterized in that: include: Obtaining answer information of the online lab report, and determining target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information; If the target question type is a first question type, the target answer information is compared with the first standard answer to obtain a comparison result, and a first score is generated according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions; If the target question type is a second question type, then calculating the similarity between the target answer information and the second standard answer using a similarity calculation formula, and generating a second score based on the similarity; the second question type includes a text-based fill-in-the-blank question; If the target question type is the third question type, constructing an initial large language model, adjusting the initial large language model using a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes a short-answer question, inputting the target answer information into the target large language model, and outputting a third score; If the target question type is a fourth question type, extracting feature information from the target answer information, generating a corresponding circuit system code based on the feature information, inputting the circuit system code into a target mixed-signal modeling language model, and outputting a fourth score; the fourth question type includes a circuit design question; the target answer information includes a circuit design diagram, and one circuit design question corresponds to multiple standard answers; generating an automatic review result according to the first score, the second score, the third score, and the fourth score; The target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model, and a key function performance indicator evaluation model. The method includes: Obtaining circuit information from the answer corresponding to the circuit design question; the circuit information includes components and algorithm modules; Calling the factory mode of the circuit description automatic generator, generating corresponding instantiation code described in a mixed-signal modeling language for each component and algorithm module based on the standard algorithm modules and standard components already in the database, and obtaining instantiated components and instantiated algorithm modules; Defining instantiated component ports or instantiated algorithm module ports through the circuit description automatic generator, and configuring port parameters; Determine the connection relationship of each instantiated component or instantiated algorithm module according to the port parameters, and obtain the circuit system code described in the target mixed-signal modeling language; Review and score the circuit code using the code review language model, and output a static score; Construct various key functional performance indicators of the circuit code through the key functional performance indicator evaluation model, run the circuit code to measure the indicators, and output dynamic scores; The static score and the dynamic score are weighted to obtain the fourth score.

2. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: Calculating the similarity between the target answer information and the second standard answer using a similarity calculation formula includes: Calculating the cosine similarity and Jaccard similarity between the target answer information and the second standard answer respectively; The similarity between the target answer information and the second standard answer is calculated according to the following similarity calculation formula: T = w × Sim + (1-w) × Jaccard ； Among them, T is similarity, Sim is cosine similarity, Jaccard is Jaccard similarity, w is weight, 0≤w≤1, w= , length is the length of the reference answer, and β is a positive coefficient.

3. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: The adjusting the initial large language model by using a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model includes: Obtaining weight parameters of the initial large language model; Freezing the weight parameters of the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain an intermediate large language model; An enhanced training dataset is obtained using a hybrid interactive data augmentation method, and the intermediate large language model is trained using the enhanced training dataset and a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain the target large language model.

4. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 3 is characterized in that: The training process of the target large language model includes: Obtaining short-answer questions and standard answers corresponding to the short-answer questions; Inputting the short-answer question and the standard answer to the short-answer question into the answer generation module of the initial large language model, and causing the answer generation module to output a plurality of derived answers corresponding to the standard answer to the short-answer question; Inputting the short-answer question and the corresponding multiple derived answers into the automatic review module of the initial large language model, and causing the automatic review module to output review results and scores corresponding to the derived answers to the short-answer question; Correcting the derived answer to the short-answer question and the corresponding review result and score to obtain a corrected derived answer to the short-answer question and the corresponding corrected review result and score; Generating multiple training data sets through instruction fine-tuning according to the short-answer question titles, the standard answers to the short-answer questions, the multiple revised derived answers, and the revised review results and scores; Inputting each of the training data sets into a secondary decomposition low-rank adaptive fine-tuning module for model training; By formula: The SVD decomposition and CUR decomposition are combined into a secondary low-rank decomposition, and the secondary low-rank decomposition process is embedded into the loss function of the secondary decomposition low-rank adaptive fine-tuning module; wherein, It is the weight parameter matrix of the large language model pre-training, which is the part that is frozen and does not participate in fine-tuning training. =BA, B∈R m×r , A∈R r×n are two low-rank matrices, r is the low-rank dimension, m and n are the dimensions of the weight matrix; The low-rank regularization function is defined by the training algorithm: represents the low-rank approximation of the matrix, Represents the low-rank approximate matrix obtained by performing SVD and CUR secondary decomposition on BA, represents the Frobenius norm; And through the formula: The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation; is the loss function in the original LoRA fine-tuned model; The low-rank matrices B and A are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence criterion, the training of the target large language model is determined to be completed.

5. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: Inputting the circuit system code into a target mixed-signal modeling language model and outputting a fourth score includes: Inputting the circuit system code into a circuit system code review language model in a target mixed-signal modeling language model to perform circuit static review, and outputting a circuit system static score corresponding to the circuit system code; Use the key functional performance indicator evaluation model to predefine the target indicators of the circuit system code, and build a target indicator calculation module in the target mixed-signal modeling language model; Using the key functional performance index evaluation model, a mixed signal modeling language simulator is used to load the circuit system code described in the mixed signal modeling language converted from the actual answer to the circuit design problem, and at the same time load the target index calculation module preset for the circuit design problem; Running the circuit system code corresponding to the actual answer to the circuit design question using the key functional performance indicator evaluation model to output a target indicator value, inputting the target indicator value into a dynamic review module of the target key functional performance indicator evaluation model to output a circuit system dynamic score corresponding to the circuit system code; The circuit system static score and the circuit system dynamic score are weighted to obtain the fourth score.

6. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: Determining target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information includes: Dividing the answer information of the online experiment report into multiple answer areas, each answer area corresponding to a question type; Determine a target answer area corresponding to the target answer information to be reviewed, and determine a target question type corresponding to the target answer information from the question types based on the target answer area.

7. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: If the target question type is the first question type, the target answer information is compared with the first standard answer to obtain a comparison result, including: If the target question type is a multiple-choice question or a true-or-false question, the target answer information is compared with the standard answer corresponding to the multiple-choice question or the fill-in-the-blank question to determine whether the comparison is successful or failed; If the target question type is a numerical fill-in-the-blank question, determining a numerical range corresponding to a standard answer to the numerical fill-in-the-blank question; The target answer information is compared with the numerical range to determine whether the target answer information is within the numerical range.

8. An automatic review system for online experimental reports of electronic information and communication majors, characterized by: include: An acquisition module is used to obtain answer information of the online experiment report, and determine the target answer information to be reviewed and the target question type corresponding to the target answer information from the answer information; a first scoring module configured to compare the target answer information with a first standard answer if the target question type is a first question type, obtain a comparison result, and generate a first score based on the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions; a second scoring module configured to calculate the similarity between the target answer information and the second standard answer using a similarity calculation formula if the target question type is a second question type, and generate a second score based on the similarity; the second question type includes a text-based fill-in-the-blank question; A third scoring module is configured to, if the target question type is a third question type, construct an initial large language model, adjust the initial large language model using a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes short-answer questions, input the target answer information into the target large language model, and output a third score; a fourth scoring module configured to, if the target question type is a fourth question type, extract feature information from the target answer information, generate a corresponding circuit system code based on the feature information, input the circuit system code into a target mixed-signal modeling language model, and output a fourth score; the fourth question type includes a circuit design question; the target answer information includes a circuit design diagram, and one circuit design question corresponds to multiple standard answers; A result generating module, configured to generate an automatic review result according to the first score, the second score, the third score, and the fourth score; The target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model, and a key function performance indicator evaluation model. The automatic review system is also used to: Obtaining circuit information from the answer corresponding to the circuit design question; the circuit information includes components and algorithm modules; Calling the factory mode of the circuit description automatic generator, generating corresponding instantiation code described in a mixed-signal modeling language for each component and algorithm module based on the standard algorithm modules and standard components already in the database, and obtaining instantiated components and instantiated algorithm modules; Defining instantiated component ports or instantiated algorithm module ports through the circuit description automatic generator, and configuring port parameters; Determine the connection relationship of each instantiated component or instantiated algorithm module according to the port parameters, and obtain the circuit system code described in the target mixed-signal modeling language; Review and score the circuit code using the code review language model, and output a static score; Construct various key functional performance indicators of the circuit code through the key functional performance indicator evaluation model, run the circuit code to measure the indicators, and output dynamic scores; The static score and the dynamic score are weighted to obtain the fourth score.

Images