Digital paper marking system for subjective test questions of test paper

Abstract

The invention relates to the technical field of artificial intelligence natural language processing, and discloses a test paper subjective test question-oriented digital paper marking system, which adopts a C / S (Client / Server) architecture to provide a physical deployment framework for a functional module, and comprises a running environment layer, a data layer, a business logic layer and a view interaction layer, the function module comprises a user login module, a user management module, an authority management module, a question bank management module, an examination management module, a test paper management module, a dictionary management module and a log management module, and the function module realizes the specific capability of the C / S architecture through logic splitting. According to the digital paper marking system for the subjective test questions of the test paper, the functions of correcting a whole set of test paper, correcting test papers of different subjects and automatically issuing scores are achieved, the problems of environmental protection caused by a traditional test, poor and unclear answer papers, long consumed time and the like are solved by adopting test paper electronization and digitalization, and automatic marking of the test papers is achieved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and natural language processing technology, and in particular to a digital marking system for subjective test questions. Background Technology

[0002] Exam paper marking is an indispensable part of the examination process and one of the most challenging aspects of all examinations. It requires significant time, financial, and human resources from schools and educational institutions. In most exam marking processes, schools and institutions separate subjective and objective questions onto answer sheets, using machines for objective questions. However, uncertainties such as illegible handwriting in subjective questions can disrupt the marking process, requiring considerable time. Furthermore, the large amount of paper required for exams, in today's environment that emphasizes environmental protection and carbon reduction, contributes to pollution and contradicts these principles. Therefore, electronic and digital exams are an excellent solution, not only solving the problem of excessive paper consumption but also reducing the need for large numbers of personnel for marking.

[0003] In existing technologies, exam papers are mainly divided into two branches: one is paper-based exams where objective questions are machine-graded and subjective questions are graded by teachers; the other is electronic exams where objective questions are graded by a system and subjective questions are graded by teachers. Both existing methods require faculty members to grade the papers. The first method is more traditional and also the most time-consuming and labor-intensive, requiring a large amount of paper for recording answers, which does not align with green environmental protection and carbon emission reduction principles. The second method, while using machine grading for objective questions, still requires teachers to spend significantly more time grading subjective questions when answers are illegible. Frequent occurrences of this will severely delay grading speed and score release time. A common problem in both methods is that teachers cannot grade papers across subjects; each teacher can only grade the subjects they teach, lacking universal applicability. When large-scale exams occur, the combination of a large number of exam papers and a short time to release scores can cause some teachers to suspend classes to focus on grading the papers, resulting in delays or disconnects between students' learning plans and teachers' teaching plans. Summary of the Invention

[0004] This invention provides a digital marking system for subjective questions in exam papers, enabling the marking of the entire set of exam papers, different subjects' exam papers, and automatic score generation. By using electronic and digital exam papers, it solves the environmental problems caused by traditional examinations, as well as the problems of illegible answers and long time consumption, and realizes the automated marking of exam papers.

[0005] This invention provides a digital marking system for subjective test questions, which adopts a client / server (C / S) architecture to provide a physical deployment framework for functional modules. The C / S architecture includes a runtime environment layer, a data layer, a business logic layer, and a view interaction layer. The functional modules include a user login module, a user management module, a permission management module, a question bank management module, an examination administration management module, a test paper management module, a dictionary management module, and a log management module. The specific capabilities of the C / S architecture are realized through logical decomposition of the functional modules.

[0006] Runtime environment layer: Runs on a server with a Linux operating system installed, and builds a Docker application container engine as the runtime environment for the system and its dependencies. Nginx is used for HTTP and reverse proxy web servers, Tomcat is used as the web server gateway interface, and Git is used for system code version updates, maintenance and automated deployment.

[0007] Data Layer: MySQL, a relational database, is used as the data storage system, and Redis is introduced as a data cache and message queue tool. The data stored in MySQL includes user basic information, question bank and test question information, exam tasks and exam paper and test question information, test paper error summary information, test paper generation rule information, dictionary basic information, log operation record information, relevant datasets obtained from model training, and relevant scheduled task information.

[0008] Business logic layer: Spring Boot, a framework based on the Java programming language, is used as the program framework for developing backend business logic functions. At the same time, the marking and testing model is set up in an independent running environment to achieve relative separation between the web front-end and business logic functions and the marking and testing model. The two services communicate and execute using a message queue triggered by Redis.

[0009] View Interaction Layer: The layer that directly interacts with the user, including the middle platform management interface based on the C / S architecture, which is jointly implemented by the Reactjs library and the Antd Pro front-end UI component library. At the same time, Hugging Face is introduced into the marking and detection model training interface to realize effective annotation of submitted sample information.

[0010] Furthermore, in the aforementioned functional modules,

[0011] Login Management Module: Provides login for PC users, determines user roles through login, and thus controls the scope of permissions that users can access; user roles include students, administrators, and teachers;

[0012] User Management Module: Enables administrators to effectively manage all users within the system to meet system operational needs;

[0013] Access control module: It satisfies the access scope of different roles to the system. It is designed based on the RBAC model and allows multiple roles to use it on a single PC. It sets the menus, pages and interfaces that users can view according to their roles.

[0014] Question Bank Management Module: Manages test questions. Different question banks and versions can be created to facilitate test question management. Teacher users can use the provided templates to batch import and export test questions from the question bank for management.

[0015] The examination administration module includes test paper generation rules, examination tasks, reference tasks, and marking functions. Among them, the test paper generation rules, examination tasks, and marking functions are for teachers to use. The examination tasks are used to statistically analyze students' exam scores. The reference tasks are for students to use. The marking function is used when students do not agree with the scores marked by the system and send a request to the teacher for remarking. Teachers can then manually mark the questions and give corresponding scores.

[0016] Exam management module: Designed for students, it includes exam management, error management, and score statistics and analysis. Students can adjust their learning based on each exam score and the reasons for their mistakes, thereby ensuring a full grasp of the knowledge.

[0017] Dictionary Management Module: Manages the dictionary fields in the system to facilitate system management for administrators;

[0018] Log management module: Manages user actions within the system, allowing users to view actions taken by administrators, teachers, and students, so that administrators can easily identify system problems during maintenance.

[0019] Furthermore, the grading detection model uses Mamba, bidirectional attention mechanism and decision tree to grade exam papers. Mamba adopts an SSM architecture design and merges the MLP block of Transformer into one block to simplify the past deep sequence model architecture and obtain an architecture design that includes Selective State Space.

[0020] Furthermore, after the marking and detection model is constructed, a set number of test question data from different subjects are collected, the reference answers and test answer data of the test questions are labeled, and the collected data is divided into training set data and test set data according to a set ratio based on the amount of data.

[0021] Set the basic configuration parameters required for the model, including the learner rate, weight decay, and smoothing coefficient of the loss function. During training, dynamically adjust the weight decay value based on the loss function curve and the complexity of model training. Dynamically set the number of epochs and plot the training and validation loss curves to observe the changes in training and validation loss. Adjust the number of epochs until the model validation loss no longer improves, then stop training the model to obtain a mature grading detection model.

[0022] The model outputs the reference answer relevance coefficient and the answer relevance data to generate a score decision tree on answer relevance, thereby outputting the matching score of the test question; after the marking detection model is trained and matured, it is deployed in the prepared running environment and communicates with the application through a message queue.

[0023] Furthermore, the training process of the marking detection model includes:

[0024] Using Hugging Face, relevant scoring terms between test answers and reference answers are labeled on the page to obtain labeled data and mask position information. After the data is labeled, it is input into the marking and detection model. In the Llama 3-based embedding layer, the input text information is converted into continuous vector data.

[0025] Vector data is transmitted to the Mamba model, and textual information features are extracted through the feature network. The extracted feature data is then normalized to a uniform format by the group normalization module inside Mamba. The data is then transmitted to the subsequent bidirectional attention mechanism layer and other layers for self-supervised multi-text comparison learning. Finally, the Mamba model outputs the learning results.

[0026] After Mamba processes the data, it outputs a similarity score to measure the similarity between different texts, a classification label for each text, and a global representation or aggregated features after comparison for each text. A decision tree is constructed using the similarity score and the test item score through loop callbacks and data feedback. The root node of the decision tree consists of the test item category, the intermediate nodes consist of similarity coefficients, and the final leaf nodes consist of the final scores matched by the similarity coefficients. After training, the decision tree is generated and the relevant training data is saved. During the grading process, based on the classification labels output by Mamba and the output of the decision tree, the final score for each test item is obtained after passing through the classification layer.

[0027] Furthermore, the bidirectional attention mechanism layer transforms the text features of text pairs into key vectors, and the key generation layer formula is:

[0028] K text_1 =W k ×F text_1 ,K text_2 =W k ×F text_2

[0029] Among them, W k To set the weights of features on the key, F text_1 and F text_2 These represent the feature data of text pairs; by transforming text features into query vectors, the query generation layer formula is:

[0030] Q text_1 =W q ×F text_1 Q text_2 =W q ×F text_2

[0031] Among them, W q To set the weights of features in the query, the text features of text pairs are transformed into value vectors. The formula for the value generation layer is as follows:

[0032] V text_1 =W v ×F text_1 V text_2 =W v ×F text_2

[0033] Among them, W v To set the weights of features in terms of values; after the above calculation operations, the attention weight matrix of each text feature between text pairs is calculated in the attention weight calculation layer, and the calculation formula is as follows:

[0034]

[0035] A new context vector is generated based on the attention weights, and the new feature representation is obtained by weighted summation, as shown in the formula:

[0036] F′ text_1 =A text_1-to-text_2 V text_2

[0037] F′ text_2 =A text_2-to-text_1 V text_1

[0038] In obtaining F′ text_1and F′ text_2 After the new feature representation is generated, it is passed to a feature concatenation layer, and the new feature representation F′ is concatenated. text_1 and F′ text_2 The components are concatenated together to form a comprehensive feature vector F. fused It is then passed to subsequent learning layers for self-supervised multi-text comparison learning.

[0039] Furthermore, in the self-supervised multi-text contrastive learning process, a contrastive loss function is introduced, which considers the contrastive relationship between an anchor sample and multiple other positive and negative samples; its formula is as follows:

[0040]

[0041] Among them, z i ,z j Let z represent the embedding of two samples in a positive sample pair. k This represents the embedding of potential negative samples, typically the entire batch or the dataset excluding z. i Embeddings of samples other than itself, sim(z) i ,z j ) / τ represents the similarity metric, where τ is the "temperature parameter" used to adjust the smoothness and scale of the distribution of embedding vector similarity, and N is the total number of samples. [k≠i] It is an indicator function used to exclude comparison pairs performed on itself.

[0042] This invention also provides a digital marking method for subjective test questions, based on the digital marking system for subjective test questions described above. The method specifically includes:

[0043] S1. Collect a set number of test questions from different subjects, and divide the collected data into training set data and test set data according to a set ratio to train the marking and detection model and obtain a mature marking and detection model.

[0044] S2. The teacher role creates exam tasks in the exam administration module, sets the attribute information of the exam tasks, combines test questions to form exam papers, and distributes the exam papers; wherein, the attribute information includes start time, end time, participating personnel, exam paper mode, composition of exam paper questions, exam subject, and difficulty level of the exam paper.

[0045] S3. The student receives their exam task. After the specified time, they enter the task execution, complete the exam, and submit the test answers. The system performs simple logical processing on the objective questions, grades them, and assigns scores to the corresponding questions. The subjective questions are sent to the grading and detection model trained in step S1 through the message queue.

[0046] S4. The marking and detection model completes the marking of subjective questions, outputs the scores of the questions, and transmits them to the application system through a message queue. After the system receives the scores of the questions, it stores the scores in the question information. After the entire paper is marked, the application system calculates the final score of the entire paper and stores the final score.

[0047] S5. In the test paper management module, the student can view the final score of the test paper, as well as the detailed information on the score or points lost. For test questions with doubts about the score or incorrect task judgment, the student can submit an error correction application and designate it for manual correction by a teacher. After the manual correction is completed, the score is recorded and transmitted to the marking detection model for learning and updating the marking detection model.

[0048] The beneficial effects of this invention are as follows:

[0049] This invention adopts a C / S architecture design, which facilitates user operation and reduces system dependence on devices, thereby reducing device diversity and ensuring system compatibility. It employs a dynamic role-based approach, integrating faculty and students into the system. Different roles result in completely different content displayed to faculty and students, meeting the needs of different roles. Exam questions are digitized and entered into the system, creating corresponding question banks. Faculty and staff can enrich and modify questions in their respective professional question banks for extraction during exam paper generation. The system has corresponding settings for exam paper generation, enabling automated one-click exam paper generation for faculty and staff, generating exam papers for each student according to the set mode. After the exam, the system will determine the student's score based on their answers and generate a set of incorrect answers for each student to review later. Furthermore, the system performs statistical analysis of student exam scores, quickly identifying student performance distribution and helping faculty adjust teaching to ensure students' mastery of knowledge. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the architecture design of the digital marking system for subjective test questions in this invention.

[0051] Figure 2 This is a schematic diagram of the functional modules of the digital marking system for subjective test questions in this invention.

[0052] Figure 3 This is a schematic diagram illustrating the usage process of the digital marking system for subjective questions in exam papers according to the present invention.

[0053] Figure 4 This is a schematic diagram of the structure of the marking and detection model in this invention.

[0054] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0056] like Figure 1 As shown, this invention provides a digital marking system for subjective test questions, which adopts a C / S architecture to provide a physical deployment framework for functional modules. The C / S architecture includes a runtime environment layer, a data layer, a business logic layer, and a view interaction layer. The functional modules include a user login module, a user management module, a permission management module, a question bank management module, an examination administration management module, a test paper management module, a dictionary management module, and a log management module. The functional modules realize the specific capabilities of the C / S architecture through logical decomposition.

[0057] Runtime Environment Layer: This system runs on a server with a Linux operating system installed, and a Docker application container engine is used as the runtime environment for the marking system and its dependencies. Nginx is used for HTTP and reverse proxy web servers, and Tomcat is used as the web server gateway interface. Git is used for system code version updates, maintenance, and automated deployment.

[0058] Data Layer: The data layer primarily uses the relational database MySQL as its data storage system, while Redis is incorporated as a data cache and message queue tool. MySQL, as the data storage system, mainly stores user basic information, question banks and test question information, exam tasks and exam paper / question information, summary information of incorrect answers, test paper generation rules, basic dictionary information, log operation records, relevant datasets obtained from model training, and related scheduled task information. Redis, as a data cache and message queue, accelerates data access, reduces database load, and, as a message queue, enables loose coupling between the application and the detection model, thereby improving system stability and reliability.

[0059] Business Logic Layer: This marking system uses Spring Boot, a framework based on the Java programming language, as the program framework for developing backend business logic functions. At the same time, the detection model is set up in an independent running environment, realizing the separation of the web front-end and business logic functions from the marking model. The two services communicate and execute using a message queue triggered by Redis, thereby improving the scalability and reusability of the system and reducing the coupling between the two services.

[0060] View Interaction Layer: The view interaction layer is the layer that directly interacts with the user. It mainly consists of two parts. The first part is the middle platform management interface based on a C / S architecture, which is mainly implemented by the React.js library and the Antd Pro front-end UI component library. At the same time, Hugging Face is introduced in the detection model training interface to realize effective annotation of submitted sample information.

[0061] This invention adopts a C / S architecture design, which facilitates user operation and reduces system dependence on devices, thereby reducing device diversity and ensuring system compatibility. The system design employs a dynamic role-based approach, integrating faculty and students. Different roles result in completely different content displayed to faculty and students, meeting the needs of different roles. Exam questions are digitized and entered into the system, creating corresponding question banks. Faculty and staff can enrich and modify questions in their respective professional question banks for extraction during exam paper generation. The system has corresponding settings for exam paper generation, enabling automated one-click exam paper generation for faculty and staff, generating exam papers for each student according to the set mode. After the exam, the system will determine the student's score based on their answers and generate a set of incorrect answers for each student to review later. Furthermore, the system statistically analyzes student exam scores, quickly identifying student performance distribution and helping faculty adjust teaching to ensure students' mastery of knowledge.

[0062] In the process of grading exam papers, the system can use simple logic processing to grade objective questions, reducing both manpower and time. For subjective questions, digitization eliminates illegible handwriting, individual differences in writing style, and the challenges of image recognition, significantly reducing workload and increasing efficiency. However, analyzing subjective questions and determining the corresponding scores remains a significant challenge. To address this, a digital grading method for subjective questions has been introduced, resolving the relationship between the question stem, the reference answer, and the answer itself, while simultaneously outputting the matching score to the answer.

[0063] This paper presents a digital grading method for subjective questions, employing Mamba, a bidirectional attention mechanism, and decision trees. Mamba combines the SSM architecture with Transformer's MLP block into a single block, simplifying the traditional deep sequence model architecture and resulting in an architecture that includes a Selective State Space. Mamba also enables parallel scanning algorithms for iterative model computation without relying on CNNs. The bidirectional attention mechanism allows the model to focus on important parts of the input data when dealing with complex problems, rather than processing all information evenly. By assigning weights based on the correlation between different elements in the input data, the model can adaptively change its focus according to different inputs or task requirements. This effectively captures these dependencies without increasing sequence length, leading to more accurate contextual understanding. The decision tree is introduced to form a relational decision tree structure from the scores generated by the bidirectional attention mechanism and the relation coefficients, enabling rapid matching of question scores during grading.

[0064] After the model architecture was completed, 5000 test questions from different subjects were collected. The reference answers and test responses were labeled, and the collected data was divided into training and testing sets in a 7:3 ratio. The Adam optimizer and a learning rate of 0.001 were used for training. The number of epochs was dynamically adjusted, and training and validation loss curves were plotted to observe changes in these losses. When the model's validation loss was still changing within a small range, the number of epochs was adjusted appropriately until the validation loss no longer improved. At this point, training was stopped, and the model was considered mature. A score decision tree based on the relevance coefficients of the reference answers and the relevance data of the test responses was generated, thus outputting the matching score for the difficulty level of the test questions.

[0065] After training, the model is deployed in the prepared runtime environment and communicates with the application via a message queue to reduce coupling between the two and improve the performance of both the system and the detection model. During automated exam grading, after completing the exam, examinees submit the entire exam paper. The system stores the submitted exam questions and answers, grades the answers to non-subjective questions using simple logic, and outputs a score for each question. Simultaneously, the application system submits the question stems, reference answers, and answers for subjective questions to the grading model via a message queue. The grading model performs grading actions such as reading, detection, and matching. The grading model ultimately outputs the scores for the matched questions and sends these scores back to the system via the message queue. The system then stores the obtained scores on the corresponding questions, thus completing the grading process.

[0066] The functional modules of this invention are as follows Figure 2 As shown, the system includes modules for user login, user management, access control, question bank management, examination administration, exam paper management, dictionary management, and log management. The overall system workflow is mainly divided into three steps: First, system administrators or teachers need to participate in step S1 to train the marking and testing model and create a complete training set, resulting in a model available to users. Second, teachers also need to participate in steps S2 and S4 to create necessary exam tasks and distribute them to students, ensuring students receive the task information and participate in the exam at the designated time. In the marking process, teachers are primarily responsible for correcting errors in the exam papers. For questions that students manually mark as incorrect, the teacher corrects them, and the judgment is re-inputted into the model for learning. Finally, the system summarizes the score data for the exam task, allowing the system to statistically analyze the relevant strength coefficients of the data to facilitate further improvement of the marking model. Third, the only steps that trainees need to participate in are as follows: S3, based on the received exam task information, log in to the system at regular intervals to participate in the exam task, carefully complete the test questions within the specified time, and submit the answers to the system.

[0067] Login Management Module: In the test grading system, the login management module primarily provides login access for PC users. Login determines user roles, thereby controlling the scope of access permissions. This system mainly includes three roles: student, administrator, and teacher.

[0068] User Management Module: There are three types of users in this system. Therefore, administrators are needed to effectively manage all users in the system to meet the needs of system operation.

[0069] Access Control Module: Access control primarily determines the access scope of different roles within the system. Designed based on the RBAC model, access control allows multiple roles to use the system on a single PC. It sets the menus, pages, and interfaces that users can view based on their roles.

[0070] Question Bank Management Module: The question bank module is mainly for managing test questions. By creating different question banks and different versions, it is convenient to manage test questions. Teacher users can use the templates provided by the system to batch import and export test questions in the question bank for management.

[0071] Examination Administration Module: This is the core module of the entire examination system, containing functions such as test paper generation rules, examination tasks, reference tasks, and marking. The test paper generation rules, examination tasks, and marking functions are for teachers. The examination tasks provide statistical analysis of student exam scores, while the reference tasks are for students. The main function of marking in the administration module is to allow students to request a re-marking when they disagree with the system's score. Teachers then manually mark the questions and assign corresponding scores.

[0072] Exam Paper Management Module: The exam paper management module is designed for students and includes exam paper management, error management, and score statistics and analysis. All three functions are designed for students, allowing them to adjust their learning based on their exam scores, errors, and other factors, thereby ensuring a thorough grasp of the knowledge.

[0073] Dictionary Management Module: Dictionary management mainly involves managing the dictionary fields existing in the system, making it easier for administrators to manage the system.

[0074] Log Management Module: This module primarily manages user actions on the system, showing when and what actions users performed. These users include administrators, teachers, students, and other roles. This facilitates system maintenance by allowing administrators to identify system issues. This feature is only available to administrators.

[0075] The system provided by this invention is designed using a client / server model. The only difference between administrators and ordinary users is that ordinary users simply use the system, while the model is trained by administrators or instructors. The model used by ordinary users is a mature, pre-trained model. The detailed process of the entire system is described below:

[0076] S1, Training Question Intelligent Grading Model

[0077] Prepare 30,000 subjective test questions (the number is set and can be adjusted according to specific needs, and is not limited here) for testing. For each question, prepare 1,000 corresponding answer sheets with different scores from different students. The scores in the 1,000 answer sheets must be evenly distributed, and all scores from 0 to full marks must be present to ensure the sufficiency of the sample and meet the comprehensiveness of model training.

[0078] On the interactive page, users submit the test questions, reference answers, test answers, scores, and subjects to be trained. Hugging Face is used to annotate relevant scoring-related words between the test answers and reference answers, obtaining annotated data and mask position information. After data annotation, the data is input into the detection model. In the Llama3-based embedding layer, the input text information is converted into continuous vector representations. High-dimensional word vectors are transformed into a low-dimensional latent space, resulting in a more compact representation and improved computational efficiency. Vectorized representations help capture semantic relationships between words, facilitating later model use. Simultaneously, the embedding layer allows words with similar meanings or contexts to have similar vector representations.

[0079] After the embedding layer, the output is text-converted vector data, which is then transmitted to the Mamba model. Mamba extracts textual features through a feature network. The extracted features are then normalized to a uniform format using Mamba's internal group normalization module before being passed to subsequent bidirectional attention layers and other layers for self-supervised multi-text comparison learning. This self-supervised multi-text comparison learning process involves a comparison loss function that considers the comparison relationship between an anchor sample and multiple other positive and negative samples. Its formula is as follows:

[0080]

[0081] Among them, z i ,z j Let z represent the embedding of two samples in a positive sample pair. k This represents the embedding of potential negative samples, typically the entire batch or the dataset excluding z. i Embeddings of samples other than itself, sim(z) i ,z j ) / τ represents the similarity metric, where τ is the "temperature parameter" used to adjust the smoothness and scale of the distribution of embedding vector similarity, and N is the total number of samples. [k≠i] It is an indicator function used to exclude comparison pairs performed on itself.

[0082] Introducing a bidirectional attention mechanism into the Mamba model allows the model to focus on key details in the text by highlighting important words or phrases with attention weights. Simultaneously, bidirectional attention can effectively identify and represent question-and-answer pairs by processing paired inputs in parallel and cross-processing, and it also becomes more accurate in understanding asymmetric relationships during contrastive learning. Bidirectional attention allows each pair of texts to influence each other, enriching the representation of one text with contextual information from the other. This interactive capability can better capture potential semantic connections or points of contrast. The bidirectional attention mechanism produces more refined and discriminative text representations, which is crucial for the subsequent generation of decision trees and also improves the overall performance of the model.

[0083] When using a bidirectional attention mechanism for data processing, the text features of text pairs are transformed into key vectors. The key generation layer formula is:

[0084] K text_1 =W k ×F text_1 ,K text_2 =W k ×F text_2

[0085] Among them, W k To set the weights of features on the key, F text_1 and F text_2 These represent the feature data of text pairs; by transforming text features into query vectors, the query generation layer formula is as follows:

[0086] Q text_1 =W q ×F text_1 Q text_2 =W q ×F text_2

[0087] Among them, W q To set the weights of features in the query, the text features of text pairs are transformed into value vectors. The formula for the value generation layer is as follows:

[0088] V text_1 =W v ×F text_1 V text_2 =W v ×F text_2

[0089] Among them, W v To set the weights of features in terms of values; after the above calculation operations, the attention weight matrix of each text feature between text pairs is calculated in the attention weight calculation layer, and the calculation formula is as follows:

[0090]

[0091] A new context vector is generated based on the attention weights, and the new feature representation is obtained by weighted summation, as shown in the formula:

[0092] F′ text_1 =A text_1-to-text_2 V text_2

[0093] F′ text_2 =A text_2-to-text_1 V text_1

[0094] In obtaining F′ text_1 and F′ text_2 After the new feature representation is generated, it is passed to a feature concatenation layer, and the new feature representation F′ is concatenated. text_1 and F′ text_2 The components are concatenated together to form a comprehensive feature vector F. fused The results are then passed to subsequent learning layers for self-supervised multi-text comparison learning, and finally the Mamba model outputs the learning results.

[0095] After Mamba processes the data, it outputs a similarity score to measure the similarity between different texts, a classification label for each text, and a global representation or aggregated features after comparison for each text. A decision tree is then constructed using the similarity score and the test item score through loop callbacks and data feedback. The root node of the decision tree consists of the test item category, the intermediate nodes consist of similarity coefficients, and the final leaf nodes consist of the final scores matched by the similarity coefficients. After training, the decision tree generation saves the relevant training data. During the grading process, based on the classification labels output by Mamba and the output of the decision tree, the final score for each test item is obtained after passing through the classification layer.

[0096] Before training the model, the basic configuration parameters required by the model need to be set, including the learner rate of the optimizer, weight decay, and the smoothing coefficient of the loss function. When setting the initial values, the learner rate of the Adam optimizer for the exam paper grading model is set to 0.001, Beta1 = 0.9, and Beta2 = 0.999 as the initial values ​​for the optimization parameters during training; the initial value of the weight decay is set to 0.0001, and the value of the weight decay is dynamically adjusted during training based on the loss function curve and the complexity of the model training. The smoothing coefficient in the loss function is 0.1. In the natural language processing model, due to the significant semantic analysis involved, the batch size is set to 256; a dropout rate of 0.1 is set; and an initial number of epochs is also set. Because of the significant semantic analysis involved in natural language processing, a large number of epochs should be set to improve the matching speed of the natural language model, enabling it to quickly improve accuracy. During training, the values ​​are adjusted accordingly to make the extracted data features more reliable. The training and validation loss curves are dynamically plotted, and their changes during training are observed in real time. The number of training epochs is adjusted appropriately. When the model validation loss curve stops changing, it means that the model's validation loss is no longer improving, and training can be stopped at this point.

[0097] S2. Create an exam task

[0098] After the grading model training is completed in step S1, teachers can create corresponding exam tasks on the exam task page of the system's exam administration module. They can select existing test paper assembly rules or create new ones, and submit them to the system. The system will then match suitable questions and assemble template exam papers based on the submitted rules. When creating an exam task, the creator needs to set the start and end times, participating personnel, exam paper mode, question composition, exam subject, and difficulty level. Once the assembly rules are set, the questions can be combined into an exam paper. After the exam paper assembly is complete, the creator needs to distribute the exam papers. Only after distribution will participants receive their exam task information and exam papers, and they can only enter and execute the task within the specified time.

[0099] S3. Take the exam and submit the exam paper.

[0100] After the exam task is successfully published in step S2, participants will receive their exam task on the exam task page, which will indicate the start and end times. Once the specified time has elapsed, users can click the button to enter the task and complete the exam. After completing the exam, users click the final submit button to end the exam. Upon receiving the submission request, the system will perform simple logical processing on the objective questions, assign scores to the corresponding questions, and send the subjective questions to the exam grading model trained in step S1 via a message queue.

[0101] S4. The marking system grades the test questions.

[0102] After step S3, the marking model receives the question stem, reference answer, answer, and category information of the subjective questions. Following the model's requirements, it inputs the question information into the model and waits for the model to complete the marking. Once the marking model has completed marking the subjective questions, it outputs the score and transmits it to the application system via a message queue. The system receives the score and stores it in the question information. After the entire exam paper is marked, the application system calculates the final score and stores it.

[0103] S5, Marking System Outputs Scores

[0104] After step S4 is completed, test takers can view their final score on the page corresponding to the test paper menu. Clicking on a test paper allows them to view detailed scores or points deducted. While reviewing their test papers, test takers can submit correction requests for questions with questionable scores or incorrect task assessments. These requests will be assigned to a teacher for manual grading. After manual grading, the graded question information and score will be recorded. If the grading is for subjective questions, the results will be fed into the test paper grading model for further learning, improving the model's understanding and accuracy.

[0105] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0106] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A digital marking system for subjective questions in exam papers, characterized in that, A C / S architecture is adopted to provide a physical deployment framework for functional modules. The C / S architecture includes a runtime environment layer, a data layer, a business logic layer, and a view interaction layer. The functional modules include a user login module, a user management module, a permission management module, a question bank management module, an examination administration module, an exam paper management module, a dictionary management module, and a log management module. The specific capabilities of the C / S architecture are realized through logical decomposition of the functional modules. Runtime environment layer: Runs on a server with a Linux operating system installed, and builds a Docker application container engine as the runtime environment for the system and its dependencies. Nginx is used for HTTP and reverse proxy web servers, Tomcat is used as the web server gateway interface, and Git is used for system code version updates, maintenance and automated deployment. Data Layer: MySQL, a relational database, is used as the data storage system, and Redis is introduced as a data cache and message queue tool. The data stored in MySQL includes user basic information, question bank and test question information, exam tasks and exam paper and test question information, test paper error summary information, test paper generation rule information, dictionary basic information, log operation record information, relevant datasets obtained from model training, and relevant scheduled task information. Business logic layer: Spring Boot, a framework based on the Java programming language, is used as the program framework for developing backend business logic functions. At the same time, the marking and testing model is set up in an independent running environment to achieve relative separation between the web front-end and business logic functions and the marking and testing model. The two services communicate and execute using a message queue triggered by Redis. View Interaction Layer: The layer that directly interacts with the user, including the middle platform management interface based on the C / S architecture, which is jointly implemented by the Reactjs library and the Antd Pro front-end UI component library. At the same time, HuggingFace is introduced into the marking and detection model training interface to realize the effective annotation operation of the submitted sample information.

2. The digital marking system for subjective test questions according to claim 1, characterized in that, In the aforementioned functional modules Login Management Module: Provides login for PC users, determines user roles through login, and thus controls the scope of permissions that users can access; user roles include students, administrators, and teachers; User Management Module: Enables administrators to effectively manage all users within the system to meet system operational needs; Access control module: It satisfies the access scope of different roles to the system. It is designed based on the RBAC model and allows multiple roles to use it on a single PC. It sets the menus, pages and interfaces that users can view according to their roles. Question Bank Management Module: Manages test questions. Different question banks and versions can be created to facilitate test question management. Teacher users can use the provided templates to batch import and export test questions from the question bank for management. The examination administration module includes test paper generation rules, examination tasks, reference tasks, and marking functions. Among them, the test paper generation rules, examination tasks, and marking functions are for teachers to use. The examination tasks are used to statistically analyze students' exam scores. The reference tasks are for students to use. The marking function is used when students do not agree with the scores marked by the system and send a request to the teacher for remarking. Teachers can then manually mark the questions and give corresponding scores. Exam management module: Designed for students, it includes exam management, error management, and score statistics and analysis. Students can adjust their learning based on each exam score and the reasons for their mistakes, thereby ensuring a full grasp of the knowledge. Dictionary Management Module: Manages the dictionary fields in the system to facilitate system management for administrators; Log management module: Manages user actions within the system, allowing users to view actions taken by administrators, teachers, and students, so that administrators can easily identify system problems during maintenance.

3. The digital marking system for subjective test questions according to claim 1, characterized in that, The grading and detection model uses Mamba, bidirectional attention mechanism and decision tree to grade test papers. Mamba adopts the SSM architecture design and merges the MLP block of Transformer into one block to simplify the past deep sequence model architecture and obtain an architecture design that includes Selective State Space.

4. The digital marking system for subjective test questions according to claim 3, characterized in that, After the marking and detection model is built, a set number of test question data from different subjects are collected, the reference answers and test answer data of the test questions are labeled, and the collected data is divided into training set data and test set data according to a set ratio based on the amount of data. Set the basic configuration parameters required for the model, including the learner rate, weight decay, and smoothing coefficient of the loss function. During training, dynamically adjust the weight decay value based on the loss function curve and the complexity of model training. Dynamically set the number of epochs and plot the training and validation loss curves to observe the changes in training and validation loss. Adjust the number of epochs until the model validation loss no longer improves, then stop training the model to obtain a mature grading detection model. The model outputs the reference answer relevance coefficient and the answer relevance data to generate a score decision tree on answer relevance, thereby outputting the matching score of the test question; after the marking detection model is trained and matured, it is deployed in the prepared running environment and communicates with the application through a message queue.

5. The digital marking system for subjective test questions according to claim 4, characterized in that, The training process of the marking detection model includes: Using Hugging Face, relevant scoring terms between test answers and reference answers are labeled on the page to obtain labeled data and mask position information. After the data is labeled, it is input into the marking and detection model. In the Llama 3-based embedding layer, the input text information is converted into continuous vector data. Vector data is transmitted to the Mamba model, and textual information features are extracted through the feature network. The extracted feature data is then normalized to a uniform format by the group normalization module inside Mamba. The data is then transmitted to the subsequent bidirectional attention mechanism layer and other layers for self-supervised multi-text comparison learning. Finally, the Mamba model outputs the learning results. After Mamba processes the data, it outputs a similarity score to measure the similarity between different texts, a classification label for each text, and a global representation or aggregated features after comparison for each text. A decision tree is constructed using the similarity score and the test item score through loop callbacks and data feedback. The root node of the decision tree consists of the test item category, the intermediate nodes consist of similarity coefficients, and the final leaf nodes consist of the final scores matched by the similarity coefficients. After training, the decision tree is generated and the relevant training data is saved. During the grading process, based on the classification labels output by Mamba and the output of the decision tree, the final score for each test item is obtained after passing through the classification layer.

6. The digital marking system for subjective test questions according to claim 5, characterized in that, The bidirectional attention mechanism layer transforms the text features of text pairs into key vectors. The key generation layer formula is as follows: K text_1 =W k ×F text_1 ,K text_2 =W k ×F text_2 Among them, W k To set the weights of features on the key, F text_1 and F text_2 These represent the feature data of text pairs; by transforming text features into query vectors, the query generation layer formula is: Q text_1 =W q ×F text_1 ,Q text_2 =W q ×F text_2 Among them, W q To set the weights of features in the query, the text features of text pairs are transformed into value vectors. The formula for the value generation layer is as follows: V text_1 =W v ×F text_1 ,V text_2 =W v ×F text_2 Among them, W v To set the weights of features in terms of values; after the above calculation operations, the attention weight matrix of each text feature between text pairs is calculated in the attention weight calculation layer, and the calculation formula is as follows: A new context vector is generated based on the attention weights, and the new feature representations are obtained by weighted summation, as shown in the formula: F′ text_1 =A text_1-to-text_2 V text_2 F′ text_2 =A text_2-to-text_1 V text_1 In obtaining F′ text_1 and F′ text_2 After the new feature representation is generated, it is passed to a feature concatenation layer, and the new feature representation F′ is concatenated. text_1 and F′ text_2 The components are concatenated together to form a comprehensive feature vector F. fused It is then passed to subsequent learning layers for self-supervised multi-text comparison learning.

7. The digital marking system for subjective test questions according to claim 5, characterized in that, In the process of self-supervised multi-text contrastive learning, a contrastive loss function is introduced, which considers the contrast relationship between an anchor sample and multiple other positive and negative samples. The formula is as follows: Among them, z i ,z j Let z represent the embedding of two samples in a positive sample pair. k This represents the embedding of potential negative samples, typically the entire batch or the dataset excluding z. i Embeddings of samples other than itself, sim(z) i ,z j ) / τ represents the similarity metric, where τ is the "temperature parameter" used to adjust the smoothness and scale of the distribution of embedding vector similarity, and N is the total number of samples. [k≠i] It is an indicator function used to exclude comparison pairs performed on itself.

8. A digital marking method for subjective questions in exam papers, characterized in that, The digital marking system for subjective test questions based on any one of claims 1-7, wherein the method specifically includes: S1. Collect a set number of test questions from different subjects, and divide the collected data into training set data and test set data according to a set ratio to train the marking and detection model and obtain a mature marking and detection model. S2. The teacher role creates exam tasks in the exam administration module, sets the attribute information of the exam tasks, combines test questions to form exam papers, and distributes the exam papers; wherein, the attribute information includes start time, end time, participating personnel, exam paper mode, composition of exam paper questions, exam subject, and difficulty level of the exam paper. S3. The student receives their exam task. After the specified time, they enter the task execution, complete the exam, and submit the test answers. The system performs simple logical processing on the objective questions, grades them, and assigns scores to the corresponding questions. The subjective questions are sent to the grading and detection model trained in step S1 through the message queue. S4. The marking and detection model completes the marking of subjective questions, outputs the scores of the questions, and transmits them to the application system through a message queue. After the system receives the scores of the questions, it stores the scores in the question information. After the entire paper is marked, the application system calculates the final score of the entire paper and stores the final score. S5. In the test paper management module, the student can view the final score of the test paper, as well as the detailed information on the score or points lost. For test questions with doubts about the score or incorrect task judgment, the student can submit an error correction application and designate it for manual correction by a teacher. After the manual correction is completed, the score is recorded and transmitted to the marking detection model for learning and updating the marking detection model.

Images