Electronic information communication professional online experiment report automatic marking method and system

Abstract

The invention provides an electronic information communication major online experiment report automatic marking method and system, and belongs to the technical field of teaching information processing, and the method comprises the steps: obtaining target answer information and a target question type; if the target question type is the first question type, comparing the target answer information with a first standard answer to generate a first score; if the target question type is a second question type, generating a second score by calculating the similarity between the target answer information and a second standard answer; if the target question type is a third question type, reviewing the target answer information through the target large language model and outputting a third score; and if the target question type is a fourth question type, extracting feature information from the target answer information, generating a corresponding circuit system code according to the feature information, inputting the circuit system code into the target mixed signal modeling language model, outputting a fourth score and generating a reviewing result. According to the invention, automatic reading of the electronic information communication professional online experiment report can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of teaching information processing, and in particular to a method and system for automatically reviewing online experimental reports of electronic information and communication majors. Background Art

[0002] Automatic marking is a technology based on natural language processing, which is usually used to mark a large number of teaching documents, such as homework and test papers, to speed up the marking of documents and improve the marking efficiency. However, the existing automatic marking technology can usually only be used to mark conventional question types, while the electronic online experiment report of the electronic communication major will involve a variety of complex question types, especially short answer questions and circuit design questions. The existing automatic marking methods usually cannot meet the marking requirements of the online experiment report of the electronic communication major. Summary of the invention

[0003] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and to provide a method and system for automatically reviewing online experiment reports of electronic information and communication majors, so as to realize automatic review of online experiment reports of electronic communication majors.

[0004] In a first aspect, the present application provides a method for automatically reviewing online experimental reports of electronic information and communication majors, comprising:

[0005] Acquire answer information of the online experiment report, and determine target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information;

[0006] If the target question type is a first question type, the target answer information is compared with the first standard answer to obtain a comparison result, and a first score is generated according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions;

[0007] If the target question type is a second question type, the similarity between the target answer information and the second standard answer is calculated by a similarity calculation formula, and a second score is generated according to the similarity; the second question type includes a text-based fill-in-the-blank question;

[0008] If the target question type is the third question type, construct an initial large language model, adjust the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm, and obtain a target large language model; the third question type includes a short answer question, input the target answer information into the target large language model, and output a third score;

[0009] If the target question type is the fourth question type, extracting feature information from the target answer information, generating a corresponding circuit system code according to the feature information, inputting the circuit system code into a target mixed signal modeling language model, and outputting a fourth score; the fourth question type includes a circuit design question;

[0010] An automatic review result is generated according to the first score, the second score, the third score, and the fourth score.

[0011] In one implementation, calculating the similarity between the target answer information and the second standard answer by using a similarity calculation formula includes:

[0012] Calculating the cosine similarity and Jaccard similarity between the target answer information and the second standard answer respectively;

[0013] The similarity between the target answer information and the second standard answer is calculated according to the following similarity calculation formula:

[0014] T = w × Sim + (1-w) × Jaccard;

[0015] Among them, T is similarity, Sim is cosine similarity, Jaccard is Jaccard similarity, w is weight, 0<=w<=1, w=1-e -β*length , length is the length of the reference answer, and β is a positive coefficient.

[0016] In one implementation, the adjusting the initial large language model by using a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model includes:

[0017] Obtaining weight parameters of the initial large language model;

[0018] Freezing the weight parameters of the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain an intermediate large language model;

[0019] An enhanced training data set is obtained using a hybrid interactive data enhancement method, and the intermediate large language model is trained using the enhanced data set and a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain the target large language model.

[0020] In one implementation, the training process of the target large language model includes:

[0021] Obtaining a short-answer question title and obtaining a standard answer to the short-answer question corresponding to the short-answer question title;

[0022] Inputting the short-answer question title and the standard answer to the short-answer question into the answer generation module of the initial large language model, and allowing the answer generation module to output a plurality of derived answers corresponding to the standard answer to the short-answer question;

[0023] Inputting the short-answer question and the corresponding multiple derived answers into the automatic review module of the initial large language model, and allowing the automatic review module to output review results and scores corresponding to the derived answers of the short-answer question;

[0024] Correcting the derived answer to the short-answer question and the corresponding review result and score to obtain a corrected derived answer to the short-answer question and the corresponding corrected review result and score;

[0025] Generating multiple training data sets through instruction fine-tuning according to the short-answer question title, the short-answer question standard answer, the multiple modified derivative answers, and the modified review results and scores;

[0026] Input each of the training data sets into a secondary decomposition low-rank adaptive fine-tuning module for model training;

[0027] By formula:

[0028] W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =W (0) +B CUR A CUR

[0029] The SVD decomposition and CUR decomposition are combined into a secondary low-rank decomposition, and the secondary low-rank decomposition process is embedded into the loss function of the secondary decomposition low-rank adaptive fine-tuning module; wherein W (0) is the weight parameter matrix of the large language model pre-training, which is the part that is frozen and does not participate in fine-tuning training, ΔW = BA, B∈R m×r , A∈R r×n are two low-rank matrices, r is the dimension of the low rank, and m and n are the dimensions of the weight matrix;

[0030] The low-rank regularization function is defined by the training algorithm:

[0031]

[0032] Represents the low-rank approximation of the matrix, CU cur R represents the low-rank approximate matrix obtained by performing SVD and CUR secondary decomposition on BA. represents the Frobenius norm;

[0033] And through the formula:

[0034] L total =L task +λ SVD-CUR R SVD-CUR (B,A)

[0035] The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation; L task is the loss function in the original LoRA fine-tuned model;

[0036] The low-rank matrices y and y are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence criterion, it is determined that the training of the target large language model is completed.

[0037] In one embodiment, the target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model, and a key functional performance indicator evaluation model, and the method includes:

[0038] The answer to the question is converted into a circuit system code described in a mixed signal modeling language by the automatic circuit description generator;

[0039] Review and score the circuit code using the code review language model, and output a static score;

[0040] Through the key functional performance indicator evaluation model, various key functional performance indicators of the circuit code are constructed, and the circuit code is run to measure the indicators and output dynamic scores.

[0041] In one embodiment, the step of converting the answer to the question into a circuit system code described in a mixed signal modeling language by the circuit description automatic generator includes:

[0042] Obtaining circuit information from the answer corresponding to the circuit design question; the circuit information includes components and algorithm modules;

[0043] Calling the factory mode of the circuit description automatic generator, generating corresponding instantiation codes described in a mixed signal modeling language for each component and algorithm module according to the standard algorithm modules and standard components already in the database, and obtaining instantiation components and instantiation algorithm modules;

[0044] Defining instantiated component ports or instantiated algorithm module ports through the circuit description automatic generator, and configuring port parameters;

[0045] According to the port parameters, the connection relationship of each instantiated component or instantiated algorithm module is determined to obtain the circuit system code described in the target mixed-signal modeling language.

[0046] In one embodiment, inputting the circuit system code into a target mixed-signal modeling language model and outputting a fourth score comprises:

[0047] Inputting the circuit system code into a circuit system code review language model in a target mixed signal modeling language model to perform circuit static review, and outputting a circuit system static score corresponding to the circuit system code;

[0048] Use the key functional performance indicator evaluation model to predefine the target indicators of the circuit system code, and build a target indicator calculation module in the target mixed signal modeling language model;

[0049] Using the key functional performance index evaluation model through a mixed signal modeling language simulator, loading the circuit system code described in the mixed signal modeling language converted from the actual answer to the circuit design question, and loading the target index calculation module preset for the circuit design question;

[0050] Using the key functional performance indicator evaluation model to run the circuit system code corresponding to the actual answer to the circuit design question, output the target indicator value, input the target indicator value into the dynamic review module of the target key functional performance indicator evaluation model, and output the circuit system dynamic score corresponding to the circuit system code;

[0051] The circuit system static score and the circuit system dynamic score are weighted to obtain the fourth score.

[0052] In one implementation, determining target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information includes:

[0053] Dividing the answer information of the online experiment report into a plurality of answer areas, each answer area corresponding to a question type;

[0054] A target answer area corresponding to the target answer information to be reviewed is determined, and a target question type corresponding to the target answer information is determined from the question types based on the target answer area.

[0055] In one implementation, if the target question type is the first question type, comparing the target answer information with the first standard answer to obtain a comparison result includes:

[0056] If the target question type is a multiple-choice question or a true-or-false question, the target answer information is compared with the standard answer corresponding to the multiple-choice question or the fill-in-the-blank question to determine whether the comparison is successful or failed;

[0057] If the target question type is a numerical fill-in-the-blank question, determining a numerical range corresponding to a standard answer to the numerical fill-in-the-blank question;

[0058] The target answer information is compared with the numerical range to determine whether the target answer information is within the numerical range.

[0059] In the second aspect, the present application provides an automatic review system for online experimental reports of electronic information and communication majors, including:

[0060] An acquisition module is used to acquire answer information of the online experiment report, and determine target answer information to be reviewed from the answer information, and a target question type corresponding to the target answer information;

[0061] A first scoring module is used for comparing the target answer information with the first standard answer to obtain a comparison result if the target question type is a first question type, and generating a first score according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions;

[0062] A second scoring module, for calculating the similarity between the target answer information and the second standard answer by a similarity calculation formula if the target question type is a second question type, and generating a second score according to the similarity; the second question type includes a text-based fill-in-the-blank question;

[0063] A third scoring module is used for constructing an initial large language model if the target question type is a third question type, adjusting the initial large language model by a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes a short-answer question, inputting the target answer information into the target large language model, and outputting a third score;

[0064] a fourth scoring module, for extracting feature information from the target answer information if the target question type is a fourth question type, generating a corresponding circuit system code according to the feature information, inputting the circuit system code into a target mixed signal modeling language model, and outputting a fourth score; the fourth question type includes a circuit design question;

[0065] The result generating module is used to generate an automatic review result according to the first score, the second score, the third score and the fourth score.

[0066] The embodiments of the present invention have the following beneficial effects:

[0067] The present invention provides a method for automatically marking online experimental reports of electronic information and communication majors. Firstly, the corresponding target question type is determined through the answer information of the experimental report, and corresponding marking rules are set according to different question types, thereby realizing automatic marking of online experimental reports of electronic information and communication majors and improving the reliability and scientificity of the marking results. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 A flowchart of an automatic review method for online experimental reports of electronic information and communication professionals provided in an embodiment of the present application;

[0070] Figure 2 A schematic diagram of a flow chart of a first question type review method provided in an embodiment of the present application;

[0071] Figure 3 A flowchart of a target large language model training method provided in an embodiment of the present application;

[0072] Figure 4 A schematic diagram of the framework structure of an automatic review system for online experimental reports of electronic information and communications majors provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0074] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0076] See also Figure 1 , Figure 1A flowchart of a method for automatically reviewing online lab reports for electronic information and communications majors provided in an embodiment of the present application is provided. The method can be used to automatically review online lab reports for electronic information and communications majors. The method includes steps S101-S106.

[0077] S101, obtaining answer information of an online experiment report, and determining target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information.

[0078] Online experiments for electronic communication majors refer to the experimental reports obtained by students using electronic communication hardware experiment boxes or virtual simulation software to conduct electronic communication experiments. The experimental reports are electronic online experimental reports. Electronic experimental reports are generally divided into five types of questions, namely: multiple choice questions, true or false questions, fill-in-the-blank questions, short answer questions, and circuit design questions. Students fill in the corresponding answer information according to different question types, and then review and score the answer information in various ways.

[0079] In order to facilitate the review, the answer information of the online experiment report is usually divided into multiple answer areas in advance, and each answer area corresponds to a question type. When reviewing, it is only necessary to determine the target answer area corresponding to the target answer information to be reviewed, and then determine the target question type corresponding to the target answer information from the question types according to the target answer area, and then determine the review strategy corresponding to the question type according to the target question type, so as to realize automatic review.

[0080] S102, if the target question type is the first question type, compare the target answer information with the first standard answer to obtain a comparison result, and generate a first score according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions and numerical fill-in-the-blank questions.

[0081] See also Figure 2 , step S102 includes steps S1021-S1023.

[0082] S1021, if the target question type is a multiple-choice question or a true-or-false question, compare the target answer information with the standard answer corresponding to the multiple-choice question or the fill-in-the-blank question to determine whether the comparison is successful or failed.

[0083] S1022: If the target question type is a numerical fill-in-the-blank question, determine the numerical range corresponding to the standard answer of the numerical fill-in-the-blank question.

[0084] S1023, comparing the target answer information with the numerical range to determine whether the target answer information is within the numerical range.

[0085] The answers to multiple-choice questions or true-or-false questions are generally fixed. Therefore, you only need to compare the target answer information filled in by the students with the standard answers. If the target answer information is the same as the marked answer, then the two are successfully compared, indicating that the student filled in the correct answer. Otherwise, it means that the student filled in the wrong answer.

[0086] For numerical fill-in-the-blank questions, the standard answer is usually a number or a numerical range. If it is a number, all expressions of the standard answer can be listed as the standard answer. For example, if the answer is 0.75, then 0.75, 3 / 4, 75%, and other multiple expressions of the answer can be used as standard answers, and then compared with the target answer information filled in by the students. If the comparison is successful, it means that the answer filled in by the student is the correct answer, and the score of the question can be recorded, that is, the score. If the answer is wrong, no score will be given. Similarly, if the standard answer is a numerical range, the standard answer can be set in the same way, and compared with the target answer information filled in by the students, and then the score can be recorded.

[0087] S103. If the target question type is the second question type, the similarity between the target answer information and the second standard answer is calculated using a similarity calculation formula, and a second score is generated according to the similarity; the second question type includes a text-based fill-in-the-blank question.

[0088] The standard answer to a text-based fill-in-the-blank question is usually a word, phrase or short sentence. In the case where the answer to a text-based fill-in-the-blank question is a word or a short phrase, Jaccard similarity can be used to compare the similarity between the student's answer and the standard answer. For short sentence or one-sentence type answers, cosine similarity can be used to convert the text into a vector to calculate the semantic similarity of the text, especially when the text uses more complex expressions or synonyms.

[0089] In order to improve the accuracy of the text-based fill-in-the-blank question marking, the present application first calculates the cosine similarity and Jaccard similarity between the target answer information and the second standard answer respectively;

[0090] Then, the similarity between the target answer information and the second standard answer is calculated according to the following similarity calculation formula:

[0091] T = w × Sim + (1-w) × Jaccard;

[0092] Among them, T is similarity, Sim is cosine similarity, Jaccard is Jaccard similarity, w is weight, 0<=w<=1, w=1-e -β*length, length is the length of the reference answer, and β is a positive coefficient used to adjust the sensitivity of w to the length of the reference answer. The value of β can be adjusted according to specific scoring requirements. The larger the value, the faster w increases with length.

[0093] In this way, when length is very small, such as 1 or close to 0. If it is close to 1, then w is close to 0, and a higher weight can be given to Jaccard similarity. As length increases, w also gradually increases, and a higher weight can be given to cosine similarity. Finally, a threshold t is set, and when the answer similarity T>t, the answer is considered correct.

[0094] S104. If the target question type is the third question type, construct an initial large language model, adjust the initial large language model through a quadratic decomposition low-rank adaptive fine-tuning algorithm, and obtain a target large language model; the third question type includes short-answer questions, input the target answer information into the target large language model, and output a third score.

[0095] Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. General large language models, such as ChatGPT-4 and Cluade3, are pre-trained with general knowledge text data. Currently, there is no large language model trained in the field of electronic communications. Therefore, in order to make the large language model more accurately analyze the correctness or score of students' answers compared to the standard answers, it is necessary to use the professional knowledge of electronic communications and the real question bank to fine-tune the pre-trained large language model.

[0096] For short-answer questions, the titles and standard answers are relatively easy to obtain, but diverse student answer data is indeed difficult to obtain. In order to achieve the accuracy and scientificity of the automatic marking results of the target large language model, the student answer data needs to be processed, such as data augmentation processing.

[0097] Data enhancement processing includes two methods:

[0098] Method 1: First, collect the short-answer questions and standard answers from the question bank. For each question, collect as many real student answers and the corresponding teacher's comments and scores as possible. These real student answers and teacher's comments can be collected before the system goes online, or gradually accumulated as training data during the system's online operation to continuously optimize the large language model's comment accuracy. Input the short-answer questions, standard answers, and real student answers into the pre-trained large language model, and let the large language model rewrite the student's answer based on the question and standard answer to generate a new student answer.

[0099] In order to ensure the diversity of generated answers, it is necessary to set the parameters for the model to generate answers: including specifying the length, complexity, relevance and correctness of the answer in the instruction prompt. For example, the length of the generated answer can be set not to exceed a certain number of words to ensure the simplicity of the answer; at the same time, the creativity and diversity of the answer can be controlled by adjusting the temperature parameter of the model. The generated answer is input into the large language model together with the question and reference answer again, and the large language model is allowed to automatically review and score the newly generated answer. Then human experts will revise the review results and scores of the large language model. In this way, a new answer and review result for the question is obtained.

[0100] Method 2: Do not input any real student answers into the large language model. Directly input the short-answer questions and standard answers in the question bank into the large language model, and let the large language model directly generate new answers based on the questions and standard answers. In this process, you also need to set the parameters for the model to generate answers. The setting method is the same as method 1. The generated answers are input into the large language model again together with the questions and reference answers, and the large language model is allowed to automatically review and score them directly. Then human experts will correct the review results and scores of the large language model. In this way, a new answer and review result for the question is obtained.

[0101] See also Figure 3 , step S104 also includes steps S1041-S1043.

[0102] S1041. Obtain weight parameters of the initial large language model.

[0103] In order to maximize the accuracy of large language model fine-tuning and reduce the cost of fine-tuning, this application proposes a new quadratic decomposition low-rank adaptive fine-tuning algorithm. This algorithm requires certain modifications to the source code and parameters of the large language model, so an open source pre-trained large language model will be used as the initial large language model, such as Llama 3-70B.

[0104] Since the open-source pre-trained large language model has a huge number of weight parameters, if new knowledge is added to the large language model, the model's weight parameters need to be adjusted through training. If the weight parameters of the entire pre-trained model are adjusted, it will be a huge task in terms of both financial and time costs. Therefore, it is necessary to freeze the pre-trained weight parameters and only train the newly added knowledge data set, merging the weight parameters trained by the newly added data set into the original weight parameters. The currently commonly used method is Low-Rank Adaptation (LoRA).

[0105] S1042. Freeze the weight parameters of the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain an intermediate large language model.

[0106] In order to further improve the training accuracy of LoRA on the newly added data set, reduce the consumption of computing resources and video memory, and save training time, this embodiment proposes a secondary low-rank decomposition method and a fine-tuning algorithm.

[0107] The core idea of ​​the LoRA method is to approximate the weight matrix through low-rank decomposition, thereby reducing the number of parameters and calculations. LoRA fine-tunes the model on downstream tasks by adding a low-rank matrix to the side of the original feature. This address matrix is ​​realized by the product of two small matrices, expressed as:

[0108] W=W (0) +ΔW=BA.

[0109] Where W (0) is the weight parameter matrix of the large language model pre-training, which is the part that is frozen and does not participate in fine-tuning training, and ΔW = BA, where B∈R m×r , A∈R r×n Are two low-rank matrices, which are low-rank dimensions, usually much smaller than the dimensions m and n of the weight matrix.

[0110] And through the formula:

[0111] L total =L task +λ SvD-CUR R SvD-CUR (B,A)

[0112] The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation; L task is the loss function in the original LoRA fine-tuned model;

[0113] The low-rank matrices y and y are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence criterion, it is determined that the training of the target large language model is completed.

[0114] The weight matrix is ​​decomposed using singular value decomposition (SVD), which is a classic low-rank decomposition method. The rank of the low-rank matrix can be controlled by adjusting the number of singular values ​​retained.

[0115] First, perform SVD decomposition on the original weight matrix ΔW:

[0116] ΔW=UΣV T

[0117] Select the first singular value and its corresponding singular vector:

[0118]

[0119] Among them U k ∈R m×k ,Σ k ∈R k×k , V k ∈R n×k By setting the k value, the rank of the low-rank matrix can be dynamically controlled, so that different modules in the model can use different ranks.

[0120] CUR decomposition is a decomposition method based on matrix columns and rows, which can better preserve the structural information of the original matrix.

[0121] Based on the preliminary SVD decomposition, CUR decomposition is performed:

[0122] Select the key c columns and r rows. There are many ways to select the key rows and columns. In order to keep the formula simple, we directly select the first c columns C and the first r rows R:

[0123] C=ΔW SVD [:,top-c columns]

[0124] R=ΔW SVD [top-r rows,:]

[0125] Calculate the intermediate matrix U cur :U cur =(C T C) -1 C T ΔW SVD R T (W T ) -1

[0126] The final CUR low-rank approximation matrix is:

[0127] ΔW CUR ≈CU cur R

[0128] where C∈R m×c , R∈R r×n , U∈R c×r .

[0129] The above low-rank approximation matrix is ​​expressed as the product of two matrices for application in the LoRA formula.

[0130] Define A and B as follows:

[0131] A CUR =U cur R

[0132] B CUR =C

[0133] therefore

[0134] ΔW CUR ≈B CUR A CUR

[0135] This gives the form required in the LoRA formula:

[0136] W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =W (0) +B CUR A CUR

[0137] S1043. Use a hybrid interactive data enhancement method to obtain an enhanced training data set, and train the intermediate large language model using the enhanced data set and a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain the target large language model.

[0138] Among them, the training process of the target large language model includes:

[0139] Obtaining a short-answer question title and obtaining a standard answer to the short-answer question corresponding to the short-answer question title;

[0140] Inputting the short-answer question title and the standard answer to the short-answer question into the answer generation module of the initial large language model, and allowing the answer generation module to output a plurality of derived answers corresponding to the standard answer to the short-answer question;

[0141] Inputting the short-answer question and the corresponding multiple derived answers into the automatic review module of the initial large language model, and allowing the automatic review module to output review results and scores corresponding to the derived answers of the short-answer question;

[0142] Correcting the derived answer to the short-answer question and the corresponding review result and score to obtain a corrected derived answer to the short-answer question and the corresponding corrected review result and score;

[0143] Based on the short-answer questions, the standard answers to the short-answer questions, the multiple revised derived answers, and the revised review results and scores, multiple training data sets are generated through instruction fine-tuning.

[0144] Then, each of the training data sets is input into a secondary decomposition low-rank adaptive fine-tuning module for model training;

[0145] By formula:

[0146] W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =W (0) +B CUR A CUR

[0147] Combining singular value decomposition and CUR decomposition into a secondary low-rank decomposition, and embedding the secondary low-rank decomposition process into the loss function of the secondary decomposition low-rank adaptive fine-tuning module;

[0148] The low-rank regularization function is defined by the training algorithm:

[0149]

[0150] To represent the low-rank approximation of the matrix, CU cur R represents the low-rank approximate matrix obtained by performing SVD and CUR secondary decomposition on BA. represents the Frobenius norm.

[0151] And through the formula:

[0152] L total =L task +λ SVD-CUR R SVD-CUR (B,A)

[0153] The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation;

[0154] The low-rank matrices B and A are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence standard, the training of the target large language model is determined to be completed. After the answer information of the short-answer questions in the online experimental report is input into the target large language model, the corresponding third score can be automatically generated.

[0155] S105. If the target question type is the fourth question type, extract feature information from the target answer information, generate a corresponding circuit system code based on the feature information, input the circuit system code into a target mixed signal modeling language model, and output a fourth score; the fourth question type includes circuit design questions.

[0156] The target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model, and a key functional performance indicator evaluation model.

[0157] The automatic circuit description generator is responsible for converting the answers to the questions into circuit codes described in the Verilog-AMS mixed-signal modeling language; the code review language model is responsible for reviewing and scoring the circuit codes and outputting static scores; and the key functional performance indicator evaluation model is responsible for constructing various key functional performance indicators for measuring circuit codes, running the circuit codes for indicator measurements, and outputting dynamic scores.

[0158] Verilog-AMS is a comprehensive mixed-signal modeling language that combines Verilog-HDL (for the digital part) and Verilog-A (for the analog part), allowing complex digital, analog and mixed-signal systems to be described in the same model.

[0159] Since the component library used by the respondent is known, the corresponding component code library will be written in advance in the circuit description automatic generator, and the algorithm module used by the respondent is also known, and the corresponding algorithm module code will be written in advance in the circuit description automatic generator. If it is a module developed by the respondent, it is also written in the Verilog-AMS development language by default and used directly in the circuit description automatic generator. Then the corresponding circuit described by the automatic generation of Verilog-AMS is only to select and connect the components and module codes in the automatic generator. The automatic generation of Verilog-AMS circuit code will mainly focus on converting these specific information into corresponding Verilog-AMS code.

[0160] Therefore, the review of circuit design drawings can be converted into the review of the Verilog-AMS codes corresponding to the circuit design drawings.

[0161] Similar to short-answer questions, if you want the big language model to more accurately analyze the correctness of students' answers or scores, you need to use the professional knowledge of electronic communications and a real question bank to fine-tune the pre-trained big language model. In addition, there is no single standard answer for design questions. There may be multiple correct circuit construction methods for a question, and the marking is more flexible and changeable. It is necessary to carefully construct a training data set to fine-tune the big language model.

[0162] First, it is necessary to perform data enhancement on the answer information corresponding to the circuit design questions. The data enhancement method is the same as that for the short-answer questions. Then, the model is trained using the answer information after data enhancement to improve the grading accuracy of the grading model corresponding to the circuit design questions.

[0163] In one embodiment, extracting characteristic information from the target answer information and generating a corresponding circuit system code according to the characteristic information includes:

[0164] Obtaining target answer information corresponding to the circuit design question, and extracting the characteristic information from the target answer information; the characteristic information includes the type, quantity, port, topology and parameters of electronic components and algorithm modules;

[0165] The characteristic information is converted into the corresponding circuit system code through a circuit description automatic generator.

[0166] First, it is necessary to parse the model, quantity, connection information, etc. of components and modules from the circuit information built by the test taker collected by the experiment box or simulation software. At the same time, verify the consistency and completeness of this information to ensure that there are no omissions or errors.

[0167] Generates the corresponding instantiation code for each component based on the module and component definitions in the existing library. This includes defining an instance of each module and passing the correct parameters to the module.

[0168] Instantiating prefabricated Verilog-AMS components and algorithm modules is an important step in taking the design from the module level to specific implementation. This includes converting the abstract module description into specific hardware description language code, which mainly includes:

[0169] Step 1: Define the module interface

[0170] Each module and component has a defined interface, including input ports, output ports, and necessary control signals. This is given in the module definition file, such as the module declaration in Verilog-AMS.

[0171] Step 2: Get module parameters

[0172] A module may require specific parameters to configure its behavior, such as the bit width of the counter, the size of the memory, etc. These parameters need to be explicitly specified when instantiating.

[0173] Step 3: Automatically generate instantiation code

[0174] Use the factory mode to instantiate each corresponding prefabricated component or module using the hardware description language according to the components and algorithm modules used in the circuit built by the test-taker. The instantiation process includes specifying the name of the component or module, the connected port, and the passed parameters.

[0175] Step 4: Parameters and port mapping

[0176] Map the actually used signals, data lines, and control lines to the ports defined by the module. This involves matching port names and data conversion (such as bit width adjustment) when necessary.

[0177] Through such steps, any prefabricated Verilog-AMS components and algorithm modules can be instantiated into a larger Verilog-AMS circuit design to form a complete system-level circuit design.

[0178] Then, based on the port connection information provided in the answer information, generate the code for connecting the modules inside the circuit described by Verilog-AMS. This involves the definition and assignment of signal lines between each instance, and it is necessary to ensure that different modules can interact correctly and transmit data and control signals. This process can be divided into several specific steps, each of which is part of the automatic code generation.

[0179] The working principle of the automatic generator of circuit description of the target mixed-signal modeling language model to generate the code for the connection between modules includes:

[0180] Step 1: Parse port connection information

[0181] First, parse the connection relationship between modules from the provided information. This is usually a pair consisting of the port name of the module and the target port name to be connected. For example, a JSON file or other format of data can be used to describe which port of which module is connected to which port of another module.

[0182] Step 2: Define the signal line

[0183] Based on the parsed connection information, define the signal lines needed to implement these connections in the Verilog-AMS design. This includes generating a unique signal name for each connection and specifying the width and type of the signal (for example, single-bit signal, multi-bit vector, clock signal, etc.).

[0184] Step 3: Generate a connection statement

[0185] Generates Verilog-AMS statements for each connection that specifies that an output port of one module is connected to an input port of another module.

[0186] Automatic review of circuit design drawings can be achieved by converting the answer information in the circuit design drawing into code described by the mixed signal modeling voice, and then inputting the code into the review module of the target circuit system code review large language model.

[0187] In one embodiment, inputting the circuit system code into a target mixed-signal modeling language model and outputting a fourth score comprises:

[0188] Inputting the circuit system code into a circuit system code review language model in a target mixed signal modeling language model to perform circuit static review, and outputting a circuit system static score corresponding to the circuit system code;

[0189] Use the key functional performance index evaluation model in the target mixed-signal modeling language model to pre-define the target indicators of the circuit system code, and build a target indicator calculation module in the target mixed-signal modeling language model;

[0190] Use the key functional performance index evaluation model to load, through the mixed-signal modeling language simulator, the circuit system code described by the mixed-signal modeling language converted from the actual answer of the circuit design problem, and at the same time load the target indicator calculation module preset for the circuit design problem;

[0191] Use the key functional performance index evaluation model to run the circuit system code corresponding to the actual answer of the circuit design problem, output the target indicator value, input the target indicator value into the dynamic marking module of the target key functional performance index evaluation model, and output the dynamic score of the circuit system corresponding to the circuit system code;

[0192] Perform weighted processing on the static score of the circuit system and the dynamic score of the circuit system to obtain the fourth score.

[0193] Specifically, the following calculation method is adopted:

[0194] Based on the code marking large language model in the large language model for marking the target circuit system code, marking and scoring the circuit designed and built by the test taker is a static evaluation of the circuit code described by Verilog-AMS. In addition to the static evaluation of the design logic of the circuit itself, it is also necessary to use the key functional performance evaluation model in the target mixed-signal modeling language model to perform dynamic evaluation on various functional performance indicators during the circuit operation process. Therefore, the combination of static evaluation and dynamic evaluation is a complete evaluation of a circuit.

[0195] Among them, the fourth score = W1 × large model score + W2 × key functional performance index 1 score + W3 × key functional performance index 2 score + …… + Wn × key functional performance index n-1 score + Wn × key part topology score, where W1 to Wn are weight parameters that can be customized by the question setter, W1 + …… + Wn = 1, 0 < Wn < 1. Among them, W1 is the weight of the static score, and W2 …… Wn are the weights of each key functional performance indicator in the dynamic score.

[0196] For the performance indicators of the circuit, different performance indicators can be determined for different circuits.

[0197] For example, for digital circuits, signal propagation delay, the time for the signal to change from low level to high level (or high level to low level), power consumption and noise capacity can be used as the main performance indicators. For analog circuits, the ratio of the amplifier's output signal to the input signal, the circuit's response to signals of different frequencies, the signal-to-noise ratio and the proportion of harmonic components in the output signal can be used as the main performance indicators. Users can also choose appropriate performance indicators based on actual conditions.

[0198] S106: Generate an automatic review result according to the first score, the second score, the third score, and the fourth score.

[0199] The comprehensive score of each question type is calculated based on the first score, the second score, the third score and the fourth score, and the comprehensive score is used as the automatic marking result, thereby realizing the automatic marking of the experimental report.

[0200] The method for automatic marking of online experimental reports for electronic information and communication majors provided in this embodiment first determines the corresponding target question type through the answer information of the experimental report, and sets corresponding marking rules according to different question types, thereby realizing automatic marking of online experimental reports for electronic information and communication majors and improving the reliability and scientificity of the marking results.

[0201] See also Figure 4 The embodiment of the present application further provides an automatic review system 400 for online experimental reports of electronic information and communication majors, including:

[0202] The acquisition module 401 is used to acquire the answer information of the online experiment report, and determine the target answer information to be reviewed and the target question type corresponding to the target answer information from the answer information;

[0203] A first scoring module 402 is used for comparing the target question answer information with the first standard answer to obtain a comparison result if the target question type is a first question type, and generating a first score according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions;

[0204] The second scoring module 403 is used to calculate the similarity between the target answer information and the second standard answer by using a similarity calculation formula if the target question type is a second question type, and generate a second score according to the similarity; the second question type includes a text-based fill-in-the-blank question;

[0205] The third scoring module 404 is used for constructing an initial large language model if the target question type is the third question type, adjusting the initial large language model by a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes a short-answer question, inputting the target answer information into the target large language model, and outputting a third score;

[0206] A fourth scoring module 405 is configured to extract feature information from the target answer information if the target question type is a fourth question type, generate a corresponding circuit system code according to the feature information, input the circuit system code into a target mixed signal modeling language model, and output a fourth score; the fourth question type includes a circuit design question;

[0207] The result generating module 406 is configured to generate an automatic review result according to the first score, the second score, the third score and the fourth score.

[0208] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0209] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0210] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for automatically reviewing online experimental reports for electronic information and communication majors, characterized in that: include: Acquire answer information of the online experiment report, and determine target answer information to be reviewed and a target question type corresponding to the target answer information from the answer information; If the target question type is a first question type, the target answer information is compared with the first standard answer to obtain a comparison result, and a first score is generated according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions; If the target question type is a second question type, the similarity between the target answer information and the second standard answer is calculated by a similarity calculation formula, and a second score is generated according to the similarity; the second question type includes a text-based fill-in-the-blank question; If the target question type is the third question type, construct an initial large language model, adjust the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm, and obtain a target large language model; the third question type includes a short answer question, input the target answer information into the target large language model, and output a third score; If the target question type is the fourth question type, extracting feature information from the target answer information, generating a corresponding circuit system code according to the feature information, inputting the circuit system code into a target mixed signal modeling language model, and outputting a fourth score; the fourth question type includes a circuit design question; An automatic review result is generated according to the first score, the second score, the third score, and the fourth score.

2. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: The similarity calculation formula is used to calculate the similarity between the target answer information and the second standard answer, including: Calculating the cosine similarity and Jaccard similarity between the target answer information and the second standard answer respectively; The similarity between the target answer information and the second standard answer is calculated according to the following similarity calculation formula: T = w × Sim + (1-w) × Jaccard; Among them, T is similarity, Sim is cosine similarity, Jaccard is Jaccard similarity, w is weight, 0<=w<=1, w=1-e -β*length , length is the length of the reference answer, and β is a positive coefficient.

3. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: The step of adjusting the initial large language model by using a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model includes: Obtaining weight parameters of the initial large language model; Freezing the weight parameters of the initial large language model through a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain an intermediate large language model; An enhanced training data set is obtained using a hybrid interactive data enhancement method, and the intermediate large language model is trained using the enhanced data set and a secondary decomposition low-rank adaptive fine-tuning algorithm to obtain the target large language model.

4. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 3 is characterized in that: The training process of the target large language model includes: Obtaining a short-answer question title and obtaining a standard answer to the short-answer question corresponding to the short-answer question title; Inputting the short-answer question title and the standard answer to the short-answer question into the answer generation module of the initial large language model, and allowing the answer generation module to output a plurality of derived answers corresponding to the standard answer to the short-answer question; Inputting the short-answer question and the corresponding multiple derived answers into the automatic review module of the initial large language model, and allowing the automatic review module to output review results and scores corresponding to the derived answers of the short-answer question; Correcting the derived answer to the short-answer question and the corresponding review result and score to obtain a corrected derived answer to the short-answer question and the corresponding corrected review result and score; Generating multiple training data sets through instruction fine-tuning according to the short-answer question title, the short-answer question standard answer, the multiple modified derivative answers, and the modified review results and scores; Input each of the training data sets into a secondary decomposition low-rank adaptive fine-tuning module for model training; By formula: W=W (0) +ΔW=W (0) +BA=W (0) +ΔW CUR =In (0) +B CUR AND CUR The SVD decomposition and CUR decomposition are combined into a secondary low-rank decomposition, and the secondary low-rank decomposition process is embedded into the loss function of the secondary decomposition low-rank adaptive fine-tuning module; wherein W (0) is the weight parameter matrix of the large language model pre-training, which is the part that is frozen and does not participate in fine-tuning training, ΔW = BA, B∈R m×r , A∈R r×n are two low-rank matrices, r is the dimension of the low rank, and m and n are the dimensions of the weight matrix; The low-rank regularization function is defined by the training algorithm: Represents the low-rank approximation of the matrix, CU cur R represents the low-rank approximate matrix obtained by performing SVD and CUR secondary decomposition on BA. represents the Frobenius norm; And through the formula: L total =L task +λ SVD-CUR R SVD-CUR (B,A) The regularization function is added to the loss function, and the optimization of the parameters of the secondary decomposition low-rank adaptive fine-tuning module is guided by back propagation; L task is the loss function in the original LoRA fine-tuned model; The low-rank matrices B and A are optimized by the gradient descent method. After multiple rounds of iterations, if the loss function reaches the convergence criterion, it is determined that the training of the target large language model is completed.

5. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: The target mixed-signal modeling language model includes a circuit description automatic generator, a code review language model and a key function performance index evaluation model, and the method includes: The answer to the question is converted into a circuit system code described in a mixed signal modeling language by the automatic circuit description generator; Review and score the circuit code using the code review language model, and output a static score; Through the key functional performance indicator evaluation model, various key functional performance indicators of the circuit code are constructed, and the circuit code is run to measure the indicators and output dynamic scores.

6. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 5 is characterized in that: The step of converting the answer to the question into a circuit system code described in a mixed signal modeling language by the automatic circuit description generator includes: Obtaining circuit information from the answer corresponding to the circuit design question; the circuit information includes components and algorithm modules; Calling the factory mode of the circuit description automatic generator, generating corresponding instantiation codes described in a mixed signal modeling language for each component and algorithm module according to the standard algorithm modules and standard components already in the database, and obtaining instantiation components and instantiation algorithm modules; Defining instantiated component ports or instantiated algorithm module ports through the circuit description automatic generator, and configuring port parameters; According to the port parameters, the connection relationship of each instantiated component or instantiated algorithm module is determined to obtain the circuit system code described in the target mixed-signal modeling language.

7. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 6 is characterized in that: The step of inputting the circuit system code into a target mixed-signal modeling language model and outputting a fourth score comprises: Inputting the circuit system code into a circuit system code review language model in a target mixed-signal modeling language model to perform circuit static review, and outputting a circuit system static score corresponding to the circuit system code; Use the key functional performance indicator evaluation model to predefine the target indicators of the circuit system code, and build a target indicator calculation module in the target mixed signal modeling language model; Using the key functional performance index evaluation model through a mixed signal modeling language simulator, loading the circuit system code described in the mixed signal modeling language converted from the actual answer to the circuit design question, and loading the target index calculation module preset for the circuit design question; Using the key functional performance indicator evaluation model to run the circuit system code corresponding to the actual answer to the circuit design question, output the target indicator value, input the target indicator value into the dynamic review module of the target key functional performance indicator evaluation model, and output the circuit system dynamic score corresponding to the circuit system code; The circuit system static score and the circuit system dynamic score are weighted to obtain the fourth score.

8. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: Determining target answer information to be reviewed from the answer information, and a target question type corresponding to the target answer information, including: Dividing the answer information of the online experiment report into a plurality of answer areas, each answer area corresponding to a question type; A target answer area corresponding to the target answer information to be reviewed is determined, and a target question type corresponding to the target answer information is determined from the question types based on the target answer area.

9. The method for automatically reviewing online experimental reports for electronic information and communication majors according to claim 1 is characterized in that: If the target question type is the first question type, the target answer information is compared with the first standard answer to obtain a comparison result, including: If the target question type is a multiple-choice question or a true-or-false question, the target answer information is compared with the standard answer corresponding to the multiple-choice question or the fill-in-the-blank question to determine whether the comparison is successful or failed; If the target question type is a numerical fill-in-the-blank question, determining a numerical range corresponding to a standard answer to the numerical fill-in-the-blank question; The target answer information is compared with the numerical range to determine whether the target answer information is within the numerical range.

10. An automatic review system for online experimental reports of electronic information and communication majors, characterized in that: include: An acquisition module is used to acquire answer information of the online experiment report, and determine target answer information to be reviewed from the answer information, and a target question type corresponding to the target answer information; A first scoring module is used for comparing the target answer information with the first standard answer to obtain a comparison result if the target question type is a first question type, and generating a first score according to the comparison result; the first question type includes multiple-choice questions, true-or-false questions, and numerical fill-in-the-blank questions; A second scoring module, for calculating the similarity between the target answer information and the second standard answer by a similarity calculation formula if the target question type is a second question type, and generating a second score according to the similarity; the second question type includes a text-based fill-in-the-blank question; A third scoring module is used for constructing an initial large language model if the target question type is a third question type, adjusting the initial large language model by a quadratic decomposition low-rank adaptive fine-tuning algorithm to obtain a target large language model; the third question type includes a short-answer question, inputting the target answer information into the target large language model, and outputting a third score; a fourth scoring module, for extracting feature information from the target answer information if the target question type is a fourth question type, generating a corresponding circuit system code according to the feature information, inputting the circuit system code into a target mixed signal modeling language model, and outputting a fourth score; the fourth question type includes a circuit design question; The result generating module is used to generate an automatic review result according to the first score, the second score, the third score and the fourth score.

Images