Automatic test paper marking method and system based on generative artificial intelligence

Abstract

The invention provides an automatic test paper reviewing method and system based on generative artificial intelligence, and relates to the technical field of artificial intelligence and educational evaluation, and the method comprises the steps: obtaining an image of a paper answer test paper, carrying out the content recognition of the processed image, and obtaining the content of an answer image; performing anomaly detection and correction on the content of the answer image through a generative artificial intelligence large model; performing feature extraction on the content of the corrected answer image; extracting key information from the corrected content of the answer image by using a generative artificial intelligence large model, and embedding the key information to form a new feature vector of the content of the answer image; calculating a cosine similarity score and a generative artificial intelligence large model prediction score; when a serious gap exists, a collaborative active learning mechanism is introduced, and the answer confidence of the examinee is evaluated; and when the confidence coefficient is higher than a given threshold value, outputting a prediction score. According to the method, the generative artificial intelligence large model technology is applied to the review process in a full cycle, and automatic test paper review is achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and educational assessment technology, and in particular to a method and system for automatic examination paper review based on generative artificial intelligence. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The wave of digitalization has promoted changes in the field of education. The automated scoring and rapid feedback functions of campus examination systems have become an important driving force for improving the efficiency of education assessment, and there is a great demand for educational informatization. At the same time, the rapid development of artificial intelligence technology, especially in the field of natural language processing (NLP), has led to an increasing application of machine learning technology in the field of education assessment.

[0004] Traditional examination marking methods mainly rely on the subjective judgment of teachers, which is not only inefficient but also susceptible to fatigue and subjectivity of the examiner. In order to improve the efficiency and accuracy of marking, researchers have begun to explore the use of machine learning technology to achieve automatic examination marking.

[0005] At present, although there have been a lot of research works on automatic grading, there are still some challenges and limitations. First, on the information input side, the accuracy of the automatic grading system depends largely on the accuracy of scanning and recognition technology, and there are still challenges in the accurate recognition of handwritten text and complex formulas. Secondly, in order to improve the accuracy of grading, the automatic grading system requires a large amount of annotated data for training, which may be time-consuming and costly in actual operation. Thirdly, compared with manual grading, the self-service grading system has shortcomings in explaining the grading results, which may affect the acceptance of the grading results by students and teachers.

[0006] Generative AI, especially ChatGPT, as a generative AI dialogue system based on a large language model, understands and interprets user requests through massive data storage and efficient design architecture, and generates "higher complexity response text" in a way that is close to human natural language.

[0007] At present, the applications of generative artificial intelligence in the field of education are increasing, but the existing education assessment system has not yet applied the generative artificial intelligence large model to the full cycle process of the review system. It still has the limitations of traditional systems in content recognition, feature extraction, text similarity comparison and score generation, resulting in poor review efficiency and accuracy. Summary of the invention

[0008] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a method and system for automatic examination paper review based on generative artificial intelligence, which uses a large generative artificial intelligence model, namely ChatGPT technology, to be applied to the review process throughout the entire cycle to achieve automatic examination paper review. By combining natural language processing and machine learning technology, using ChatGPT to assist in OCR recognition, enhance feature extraction and text similarity comparison, and combining with a collaborative active learning mechanism, the scoring model is continuously optimized, the review efficiency and accuracy are improved, and the introduction of an attention mechanism enhances the interpretability of the scoring results, providing efficient and accurate scoring and feedback, thereby improving the quality and efficiency of educational assessment.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of the present invention provides a method for automatic examination paper marking based on generative artificial intelligence.

[0010] A method for automatic examination paper review based on generative artificial intelligence, comprising: Acquire an image of a paper answer sheet, and perform preprocessing to obtain a processed answer image; Performing content recognition on the processed answer image to obtain the content of the answer image; performing abnormality detection on the content of the answer image through a generative artificial intelligence large model, and correcting the content of the answer image after obtaining the detection result to obtain the corrected answer image content; Perform feature extraction on the content of the corrected answer image to obtain a feature vector of the answer image content; use the generative artificial intelligence big model to extract key information from the content of the corrected answer image and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content; The cosine similarity score is calculated for the feature vector of the answer image content; at the same time, the feature vector of the answer image content is also predicted and scored through the generative artificial intelligence model; When there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than the given threshold, the predicted score is output.

[0011] A second aspect of the present invention provides an automatic examination paper marking system based on generative artificial intelligence.

[0012] An automatic examination paper review system based on generative artificial intelligence, comprising: The test paper acquisition module is configured to: acquire an image of a paper answer paper, and perform preprocessing to obtain a processed answer image; The content recognition module is configured to: perform content recognition on the processed answer image to obtain the content of the answer image; perform abnormality detection on the content of the answer image through the generative artificial intelligence large model, and correct the content of the answer image after obtaining the detection result to obtain the corrected answer image content; The feature extraction module is configured to: extract features from the content of the corrected answer image to obtain a feature vector of the answer image content; extract key information from the content of the corrected answer image using a generative artificial intelligence large model, and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content; The prediction and scoring module is configured to: calculate the cosine similarity score of the feature vector of the answer image content; at the same time, the feature vector of the answer image content is also predicted and scored through the generative artificial intelligence big model; The confidence assessment module is configured as follows: when there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than a given threshold, the predicted score is output.

[0013] The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method as described in the first aspect of the present invention are implemented.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a method as described in the first aspect of the present invention.

[0015] A fifth aspect of the present invention provides a computer program product comprising instructions, which, when running on a computer, enables the computer program to implement the steps in a method as described in the first aspect of the present invention when executed by a processor.

[0016] One or more of the above technical solutions have the following beneficial effects: The present invention applies the generative artificial intelligence big model, namely ChatGPT big model, to the review process in its entire cycle, including content recognition, feature extraction, text similarity comparison and score generation, overcoming the limitations of traditional systems. In addition, the present invention uses ChatGPT to assist OCR recognition, enhance feature extraction and text similarity comparison, and combines the collaborative active learning mechanism to continuously optimize the scoring model, improve the review efficiency and accuracy, and introduce the attention mechanism to enhance the interpretability of the scoring results.

[0017] The present invention integrates a generative artificial intelligence large model, namely the ChatGPT large model, with a variety of algorithmic technologies, using ChatGPT to assist BPD and SideNet models for OCR recognition, and combining ChatGPT for anomaly detection; ChatGPT optimizes Word2Vec, BERT, LaMDA and other algorithms for feature extraction; ChatGPT enhances text vector representation and calculates cosine similarity; and ChatGPT predicts scoring and combines active learning mechanisms for scoring decisions. The final score is obtained by calculating cosine similarity and ChatGPT's predicted scoring, and combining weight factors for weighted averaging, making the scoring results more transparent and explainable, which helps students and teachers understand and accept the scoring results.

[0018] The automatic examination paper / homework review system of the present invention based on the generative artificial intelligence large model ChatGPT has significant practicality and can be widely used in the field of education. It overcomes the problem that the traditional examination paper review method is highly dependent on manual feature extraction or data collection, greatly improves the review efficiency through the automated process, reduces labor costs, and uses ChatGPT's powerful language comprehension ability to accurately understand the text content and perform semantic analysis, thereby improving the accuracy and reliability of scoring. And the shortcomings of the self-service scoring system in explaining the scoring results, thereby achieving more efficient, accurate and explainable examination paper review. It can be widely used in scenarios such as schools, training institutions, and online education platforms, and has important practical value.

[0019] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0021] Figure 1 This is a flow chart of a method for automatic examination paper marking based on generative artificial intelligence according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0022] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0023] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0024] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0025] Embodiment 1 This embodiment discloses a method for automatic examination paper grading based on generative artificial intelligence, and proposes an automatic examination paper grading method based on a generative artificial intelligence big model, full-cycle ChatGPT and expert knowledge collaboration. The ChatGPT big model is applied to multiple key links such as answer content recognition, feature extraction, text similarity comparison and score generation. In this way, an attempt is made to use big model technology to run through and combine relevant methods in the fields of pattern recognition, multimodal learning, natural language processing, deep learning, active learning, etc. to achieve automatic scoring of examination papers, including: Acquire an image of a paper answer sheet, and perform preprocessing to obtain a processed answer image; Performing content recognition on the processed answer image to obtain the content of the answer image; performing abnormality detection on the content of the answer image through a generative artificial intelligence large model, and correcting the content of the answer image after obtaining the detection result to obtain the corrected answer image content; Perform feature extraction on the content of the corrected answer image to obtain a feature vector of the answer image content; use the generative artificial intelligence big model to extract key information from the content of the corrected answer image and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content; The cosine similarity score is calculated for the feature vector of the answer image content; at the same time, the feature vector of the answer image content is also predicted and scored through the generative artificial intelligence model; When there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than the given threshold, the predicted score is output.

[0026] In this embodiment, the generative artificial intelligence big model is specifically the ChatGPT big model.

[0027] In order to explain this embodiment more clearly, Figure 1 As shown in the figure, the implementation process of an automatic examination paper review method based on generative artificial intelligence can be specifically described as follows: Step 1: First stage: content recognition of the answer image.

[0028] Step 1-1, data collection and image segmentation: obtain the image of the paper answer sheet and perform preprocessing to obtain the processed answer image.

[0029] (1) The specific steps for obtaining the image of the paper answer sheet are as follows: Image acquisition: Obtain images of paper test papers through scanners, digital cameras and other devices.

[0030] In this embodiment, the image of the paper answer sheet obtained specifically includes: the student's handwritten answer image and the reference standard answer image.

[0031] (2) Preprocess the image of the paper answer sheet as follows: Image denoising: Remove noise from test paper images, such as salt and pepper noise, Gaussian noise, etc., to improve image quality.

[0032] Image binarization: Convert color or grayscale images into black and white binary images to facilitate subsequent test segmentation and character recognition.

[0033] Tilt correction: Detect the tilt angle of the image and make corrections to ensure accurate segmentation of test questions and correct recognition of characters.

[0034] Image segmentation: The preprocessed test paper images are segmented into individual test question images according to the content, so that the test question images can be scored one by one. The test paper images are segmented into multiple answer sub-images using the connected region method, key character segmentation method or superpixel segmentation method, each sub-image corresponds to a test question and the student's answer.

[0035] Step 1-2: Perform content recognition on the processed answer image to obtain the content of the answer image. Perform abnormality detection on the content of the answer image through the generative artificial intelligence large model, and correct the content of the answer image after obtaining the detection result to obtain the corrected answer image content.

[0036] Step 1-2-1: Use OCR technology to perform content recognition on the processed answer image.

[0037] In this embodiment, a large model-driven Optical Character Recognition (OCR) technology is used to perform content recognition on the processed answer image. The OCR technology aims to use a large-scale machine learning model to improve the accuracy and efficiency of text recognition in images.

[0038] In this embodiment, the machine learning models used by the OCR technology specifically include a BPD (Tree-based Model with Branch Parallel Decoding, BPD) model, a SideNet model, and a variety of image segmentation models.

[0039] In this embodiment, the processed answer image content includes mathematical formulas, text and sub-images.

[0040] For example, the answer image of a math test will contain sub-images, math formulas, text, etc. Therefore, OCR technology is used to recognize the answer content, including: (1) Mathematical formula recognition For the mathematical formulas in the processed answer images, the BPD (Tree-based Model with BranchParallel Decoding, BPD) model is used to perform mathematical formula recognition.

[0041] The BPD model can effectively recognize mathematical formula structures and use the pre-trained Transformer model to predict symbols and relationships. The core idea of ​​the model is to reduce the number of decoding time steps through a parallel decoding mechanism, thereby improving recognition efficiency and accuracy.

[0042] By using the BPD model, various types of mathematical formulas can be accurately identified, such as basic operation symbols such as addition, subtraction, multiplication, and division, brackets, fractions, exponents, trigonometric functions, logarithms, etc. The operation symbol model can identify these operators and match them with the corresponding operands.

[0043] (2) Text recognition For the text (non-mathematical formulas) in the processed answer image, the SideNet model is used for OCR recognition to convert the text in the image into text data.

[0044] The SideNet model can effectively recognize Chinese characters by jointly learning radicals and glyph information, solve the problem of repeated mapping, and improve recognition accuracy.

[0045] The main components of the SideNet model include: CNN backbone network: extracts visual features from input images and glyphs.

[0046] Dimensional Decomposition Conversion Module (DDCM): extracts structural information from the radical tree and converts it into structure-aware embeddings. This module decomposes the radical information into three dimensions: category, depth, and position through dimensionality decomposition, and uses learnable parameters and nonlinear fusion strategies to enhance the embedding ability.

[0047] Count-based Spatial Transformer Module (CSCM): Learns the distribution of radicals in glyph images and generates spatially aware embeddings. This module accurately encodes the position information of radicals and solves the problem of repeated mapping by predicting the radical category on the glyph feature map and using counting loss for supervision.

[0048] Two-stage classifier: Integrates radical and glyph information and classifies characters. The first stage uses a similarity map to guide feature fusion, converts the similarity map into a mask for scaling side information features, and uses residual convolution to model the similarity map and add it to the fusion process. The second stage uses the Transformer decoder layer to calculate feature correlations and outputs the final probability matrix to recognize unseen characters.

[0049] (3) Sub-graph identification and segmentation For the sub-images in the processed answer image (i.e., the test paper contains image elements), image segmentation methods such as the connected component method, the key character segmentation method or the superpixel segmentation method are used to segment the image elements in the test paper image and extract the image feature representation.

[0050] Step 1-2-2: Use the generative artificial intelligence model to detect anomalies in the content of the answer image.

[0051] Generative AI Big Model (ChatGPT) anomaly detection, as follows: For the text and formula information on the test paper, using only OCR technology may cause errors when converting the image. Therefore, before performing feature processing, it is necessary to process the outliers that appear after OCR technology recognition.

[0052] First, the confidence scoring technology provided by the OCR software is used to obtain the uncertain text area after OCR recognition for the content of the answer image, and the low-confidence text is identified.

[0053] Secondly, for the recognized text with low confidence, the ChatGPT large model is used to detect abnormal words or phrases in the OCR output, and further manual review or automatic correction is performed.

[0054] More specifically, the ChatGPT model is used to detect abnormal words or phrases in the OCR output. The steps are as follows: Use the API interface provided by ChatGPT for communication.

[0055] Before sending the request, clean the OCR-recognized text, remove extra spaces and special characters, and ensure that the text encoding is compatible with the ChatGPT API.

[0056] The OCR-recognized text data is encapsulated into JSON format and sent to the ChatGPT server via an HTTPS request. The request includes the OCR text, context information, and any specific instructions to guide ChatGPT to perform anomaly detection.

[0057] ChatGPT analyzes the text after OCR recognition to detect grammatical errors, incoherent sentence structure or content irrelevant to the topic. It also includes identifying words or phrases that OCR may have misrecognized, which are usually inconsistent with the context.

[0058] The following is an example process: Suppose OCR outputs a piece of text: "10+6= / 6." This text is sent to the ChatGPT API with a request to detect possible anomalies.

[0059] ChatGPT returns the analysis result, indicating that " / " may be a wrong recognition and the correct symbol may be "1". The system can then mark the symbol for further manual review or automatic correction, and finally obtain the corrected answer image content.

[0060] ChatGPT not only improves the quality of OCR text, but also enhances the accuracy and reliability of the entire automatic examination marking system.

[0061] Step 2, Part 2: Word segmentation and knowledge graph construction.

[0062] Knowledge graphs can dig out potential rules and trends hidden in a large amount of information and provide support for decision-making. Taking mathematics courses as an example, building a knowledge graph for it can systematically organize various knowledge points, theorems, formulas and their interrelationships in the field of mathematics in a structured form. Teachers can clearly obtain the mathematical knowledge system, check the key points of knowledge, and intuitively discover the relationship between knowledge points. Introducing knowledge graphs in the automatic marking system can not only improve the quality of feature extraction, but also improve the interpretability of automatic scoring.

[0063] Step 2-1: Keyword Extraction.

[0064] Identify important words or phrases from the answer text, which usually represent the theme or main content of the text.

[0065] In this embodiment, Jieba word segmentation and TF-IDF technology are used to extract keywords. The specific steps are as follows: (1) First, we need to calculate the word frequency (TF): for each word, calculate how often it appears in the text.

[0066] (2) Then calculate the inverse document frequency (IDF): for each word, calculate its rarity in the entire document set.

[0067] (3) Then calculate the TF-IDF score: For each word, its TF-IDF score is the product of TF and IDF. Finally, the word with the highest score is selected as the keyword. This keyword may contain the knowledge points of these questions. We have already performed text embedding before keyword extraction, so the keywords obtained are in the form of fixed-dimensional vectors.

[0068] (4) In order to identify keywords related to knowledge points, the attention mechanism of the ChatGPT model is used to analyze the importance of each word in the sentence.

[0069] The ChatGPT model can determine the relevance of a word to a knowledge point based on its contextual information. With its powerful language understanding capabilities, ChatGPT can identify key semantic elements that may be overlooked by traditional TF-IDF methods, thereby improving the accuracy and comprehensiveness of keyword vector representation. We use ChatGPT to perform semantic analysis on these keywords to ensure that they are closely related to the knowledge point and verify whether the keywords extracted by the TF-IDF algorithm are relevant to the topic of the text. The specific process is as follows: ① Reconstruction of input sentences First, the mask mechanism in self-supervised learning is used to randomly mask some words in the student’s answer text to obtain a new sentence. At the same time, the masked words are used in the prediction output of ChatGPT.

[0070] ②Model fine-tuning Next, the reconstructed sentence is passed to ChatGPT. Specifically, the API interface provided by ChatGPT is used for communication and sent to the ChatGPT server via HTTPS request. The request includes the reconstructed sentence text and the masked words to guide ChatGPT to perform self-supervised fine-tuning training.

[0071] Step 2-2, Knowledge Graph Construction.

[0072] In step 2-1, we obtain the keywords (i.e. entities) in the test questions. For example, for math test papers, we can extract mathematical concepts, formulas, theorem names, etc., and use ChatGPT to perform semantic analysis on these keywords to ensure that they are closely related to knowledge points and identify the relationships between entities, such as mathematical relationships such as "equal to", "greater than", "less than", or "definition", "derivation", etc., as well as the logical relationships between concepts. With this information, we can build relevant knowledge graphs.

[0073] The specific construction process is as follows: (1) Knowledge Representation and Fusion Represent the extracted entities and relationships in the form of triples, and use entity linking, data alignment, etc. to align and integrate the entities and relationships from different test questions, eliminating redundancy and conflicts.

[0074] (2) Text generation Use generative models to assist in generating labeled data for training information extraction models, or generate variants of math test questions to enrich the content of the knowledge graph. Use generative models (such as GPT) to deeply understand math test questions and extract implicit knowledge points and logical relationships.

[0075] (3) Knowledge graph construction Use graph construction tools such as Neo4j and RDF4J to store and manage the extracted entities and relationships, and use visualization tools to display the knowledge graph for teachers to understand and analyze.

[0076] Step 3, Part Three: Feature extraction.

[0077] After obtaining the content information of the corrected answer image, feature extraction is required for text analysis. Feature extraction is one of the key steps in this embodiment. In this embodiment, for the recognized formulas, strings are used to describe and represent them without additional feature extraction. For the recognized text and subgraphs, combined with the knowledge graph constructed in Step 2, various methods are used to extract their features, including but not limited to related technologies such as Word2Vec, Bert, and ResNet, and the generative artificial intelligence large model is used for semantic analysis to refine and adjust the features of the text information.

[0078] In this embodiment, the content of the corrected answer image is divided into text and non-text elements such as charts, graphs, or handwritten notes in the test paper.

[0079] Step 3-1: Extract features from the text content of the corrected answer image to obtain the feature vector of the text content of the answer image.

[0080] The following are the steps for feature extraction from the text content: (1) The first step to be carried out is tokenization: decompose the text content of the answer image into words or phrases (tokens) and remove common but insignificant words in the text, such as "de", "he", etc.

[0081] (2) Then construct a window: for each word, select the surrounding words to form a context window.

[0082] (3) After word segmentation and context window construction, ChatGPT is used to analyze the context of each word to generate richer semantic representations, which helps the Word2Vec model capture deeper semantic relationships.

[0083] (4) The Word2Vec model is then trained to capture the semantic relationship features between words and converted into a vector representation of fixed dimension.

[0084] (5) Next, the pre-trained BERT model is used to perform sentence encoding on the candidates’ answers and standard answers, obtain sentence-level semantic relationship features, and convert them into vector representations of fixed dimensions.

[0085] BERT (Bidirectional Encoder Representations from Transformers, BERT) is a pre-trained language model based on Transformer. The BERT model can effectively capture the semantic information in a sentence and convert it into a vector of fixed dimension.

[0086] (6) Based on the feature vector representation of the semantic relationship between words captured by the Word2Vec model and the sentence-level semantic relationship feature vector representation obtained by the pre-trained BERT model, the feature vector of the text content of the answer image is obtained.

[0087] In this embodiment, the Word2Vec model is trained to capture the semantic relationship between words, specifically using the Word2Vec model for text embedding: Convert the original text into a vector representation of fixed dimension, preserving the semantic information of the text.

[0088] There are two models in the Word2Vec model, Skip-Gram and CBOW models, each of which has its own objective function. The objective function defines the learning goal of the model, that is, by adjusting the model parameters (such as word vectors), the value of the objective function is minimized. In Word2Vec, the optimization process of the objective function is the process of learning word vectors. The value of the objective function can be used to evaluate the quality of the model. During the training process, if the value of the objective function decreases, it usually means that the model performance is improving. The objective function of the Skip-Gram model is expressed as follows: (1) in is a given central word Background words The conditional probability of is the total number of training samples, t is the subscript index variable, j is the subscript index variable, is the size of the context window. θ represents the model parameters to be learned, and the J(θ) function represents the given central word The conditional probability of the context word under the condition, the purpose of optimization is to maximize this probability. In actual operation, the conditional probability It is usually calculated by the softmax function: (2) W is the size of the vocabulary, represents a subscript index variable, Indicates The vector representation corresponding to each word. and They are and The vector representation of .

[0089] The objective function of the CBOW model is expressed as follows: (3) is the total number of training samples, It is the central word. Is the central word Background word collection is the background word set Next central word The conditional probability of .

[0090] Similar to formula (2), the conditional probability These objective functions are also calculated through the softmax function and minimized during the training process through optimization algorithms such as gradient descent, so as to learn the effective vector representation of words.

[0091] Step 3-2: Use the generative artificial intelligence model to extract key information from the text content of the corrected answer image, and embed it into the feature vector of the answer image text content to form a new feature vector of the answer image text content.

[0092] In this embodiment, in order to further improve the quality of semantic representation, ChatGPT is used to optimize the BERT sentence encoding result (i.e., the feature vector of the answer image content) obtained in step 3-1. ChatGPT, through its powerful language understanding ability, can identify the key information in the sentence and integrate it into the semantic representation, so that the semantic representation more accurately reflects the semantic information of the answer. The process of optimizing the BERT sentence encoding result using ChatGPT is as follows: ① Reconstruction of input sentences First, the text content of the answer image is reconstructed as an input sentence, and the text content of the answer image (including the student's answer and the reference standard answer) is converted into a more structured format.

[0093] This process is accomplished through the following steps: Combine the student's answer with the corresponding knowledge concepts (KCs) and question text in the knowledge graph built in step 2 so that ChatGPT can understand the context. Reorganize the input to ensure that key information is included, such as the question statement, the student's answer content, etc.

[0094] ②Semantic information extraction Next, the text content of the reconstructed answer image is passed as input to ChatGPT. Through this process, ChatGPT uses its powerful natural language understanding capabilities to extract semantic information, including: Identify key knowledge points mentioned in the answer; Evaluate the plausibility and accuracy of students’ responses based on the context of the question; Assign a weight to each knowledge concept being tested, reflecting its importance in the answer.

[0095] ③Embedded Integration After extracting the semantic information of key information from ChatGPT, these semantic information are embedded into the feature vector of the answer image content (that is, integrated with the sentence encoding result of BERT), the semantic embedding generated by ChatGPT is concatenated with the corresponding sentence representation of BERT, and the two representations are integrated together through a simple linear layer to finally form a more effective sentence embedding representation, forming a new vector that incorporates rich semantic information.

[0096] ④Model fine-tuning We fine-tune the model using sentence features from ChatGPT output to improve performance on specific review tasks. We train on new datasets using gradient descent to optimize model parameters so that it can better capture the characteristics of diverse student responses.

[0097] Through the above steps, the powerful language understanding ability of ChatGPT and the sentence encoding ability of BERT are combined to optimize the semantic representation of students' answers in the automatic examination marking system.

[0098] Step 3-3: extract features of the non-text content of the corrected answer image to obtain a feature vector of the non-text content of the answer image.

[0099] In this embodiment, the feature extraction of non-text content is specifically embodied as image feature extraction (if the test paper contains image elements): For non-text elements such as charts, graphs, or handwritten notes in the test paper, we use the pre-trained image recognition model ResNet (Residual Network, ResNet) to extract visual features. The image data is input into the ResNet model, and after multiple convolutional layers and pooling layers, a series of high-dimensional feature vectors are extracted. The feature map corresponding to each image is obtained, usually the output of the last layer, as the visual representation of the image.

[0100] Step 3-4: Use the generative artificial intelligence large model to extract key information from the non-text content of the corrected answer image, and embed it into the feature vector of the non-text content of the answer image to form a new feature vector of the non-text content of the answer image.

[0101] In this embodiment, the non-text content is specifically embodied as image elements contained in the answer image.

[0102] In order to better understand the semantic information in the image, we use ChatGPT to perform semantic analysis on image features, such as identifying objects or symbols in the image. ChatGPT can associate image features with text information through its powerful language understanding ability, so as to better understand the semantic information of the image. The process of using ChatGPT to optimize ResNet to extract visual features is as follows: (1) Feature reconstruction Since visual features alone may not provide enough contextual information, we combine image features with answer text so that ChatGPT can fully understand the relationship between the image content and the student's answer. Therefore, the image features are reconstructed and relevant text information is integrated, such as the description corresponding to the image, the question requirements, etc., to form a richer input format.

[0103] (2) Semantic information enhancement The answer image content reconstructed from image features is passed to ChatGPT. Through ChatGPT's language generation and comprehension capabilities, a high-level semantic description is generated based on image features and text information, and improved semantic information is extracted. This helps reveal the meaning of key elements in the image. It also identifies features in the image that are relevant to the question and evaluates the relevance and rationality of these features in the student's answer.

[0104] (3) Embedded Integration After extracting the improved semantic information from ChatGPT, the enhanced visual features are integrated with the original features extracted by ResNet to take advantage of the advantages of both: the semantic embedding generated by ChatGPT is concatenated with the visual features extracted by ResNet to form a richer representation vector.

[0105] Linear transformation and standardization are performed to ensure that the dimensions of the integrated features are consistent in preparation for subsequent model training.

[0106] Step 4, Part 4: Score prediction and output.

[0107] In this embodiment, in order to improve the accuracy and effectiveness of the generated score, a score generation module of collaborative active learning and ChatGPT is designed, including: score prediction based on text similarity, score prediction based on ChatGPT, consistency detection and score correction.

[0108] Step 4-1: Calculate the cosine similarity score for the feature vector of the answer image content.

[0109] In this embodiment, the feature vector of the answer image content specifically includes: a standard answer text vector and an answer text vector.

[0110] In this embodiment, the cosine similarity score is calculated for the feature vector of the answer image content, specifically, the cosine similarity between the standard answer text vector and the answer text vector is calculated to obtain a score prediction based on text similarity.

[0111] That is, after extracting features from student answers and standard answers, the features of student answers are compared with the features of reference answers to measure the similarity between the two.

[0112] In this embodiment, the large model text similarity comparison and large model knowledge point matching rate of this process are obtained by calculating the cosine similarity. The specific steps are as follows: (1) Calculate the cosine similarity between the standard answer text embedding vector A and the answer text embedding vector B. The calculation formula for cosine similarity is: (4) Among them, A and B are the vector representations of the reference answer and the candidate's answer text respectively. And ||A|| and ||B|| are the Euclidean norms of vectors A and B respectively, that is, the lengths of the vectors. The Euclidean norm calculation formula is as follows (5) Among them, i is the subscript index variable, n represents the length of vector A, are the components of vector A.

[0113] After the calculation is completed, a cosine similarity value will be obtained. The value range of cosine similarity is between [-1, 1], where 1 means that the two vectors are exactly the same, that is, the student's answer and the standard answer are exactly the same; 0 means that the two vectors are orthogonal (that is, unrelated), and -1 means that the two vectors are completely opposite, that is, the student's answer and the standard answer are unrelated (the answer is irrelevant).

[0114] In this embodiment, considering that a higher cosine similarity score indicates that the examinee's answer is closer to the reference answer in terms of text content, the score based on cosine similarity is defined as follows: If cosine_similarity(A,B) is greater than 0, then Score_Cos=Toal_Score×cosine_simila, otherwise: Score_Cos=0.

[0115] Step 4-2: The feature vector of the answer image content is also used to obtain a predicted score through the generative artificial intelligence model.

[0116] The specific steps for score prediction based on ChatGPT are as follows: ① Collect the feature vectors of the student's answer and the standard answer, use the API interface provided by ChatGPT to communicate, and send them to the ChatGPT server through HTTPS request. Use ChatGPT's language understanding ability to analyze the relationship between them, and generate a predicted score result based on the context, existing language patterns, and knowledge related to the specific task (the knowledge graph built in step 2). The predicted score result is a specific value between 0 and 10. ②Supervised fine-tuning The actual scores of a large number of students answering test questions are passed to ChatGPT to guide ChatGPT to perform self-supervised fine-tuning training. Its parameters are adjusted according to the labeled data (answer feature vector, standard answer feature vector, true score) to obtain the trained ChatGPT score generation model.

[0117] ③Test sample score generation The embedded representation of the test questions answered by the student samples to be tested and the corresponding standard answer representation are input into the trained ChatGPT score generation model to obtain the predicted score Score_GPT.

[0118] Step 4-3: When there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than the given threshold, the predicted score is output.

[0119] That is, consistency detection and score correction. The specific steps are as follows: In case there is a serious disagreement between the cosine similarity score and the predicted score of the generative artificial intelligence large model (at this time, the confidence of the predicted score is low), an active learning mechanism is introduced to further optimize the generation process of scoring decisions and improve the accuracy and reliability of scoring. For example, effective judgment can be made on the situation where "students provide a lot of answers but get low scores".

[0120] The following are the steps for collaborative active learning scoring decision generation: (1) Confidence Assessment For the query strategy framework of active learning, we choose uncertainty sampling query (UncertaintySampling). The query method of uncertainty sampling is to extract samples that are difficult to distinguish in the model (or there are large differences in the prediction results obtained by different prediction methods), and provide them to business experts or labeling personnel for labeling, so as to achieve the ability to improve the algorithm effect at a faster speed. The key to the uncertainty sampling method is how to describe the uncertainty of samples or data. The present invention selects the idea of ​​whether the confidence meets the given threshold, that is, first calculates the confidence of the prediction score. If the confidence meets the given threshold (assuming 0.8), it is deterministic sampling, otherwise uncertainty sampling will be used in subsequent active learning modules. The confidence of the scoring result can be calculated using the prediction scores obtained by two different methods. The specific scheme is as follows: Combine the cosine similarity score and ChatGPT prediction score to evaluate the confidence of the score, represented by the symbol C: (6) Among them, the Abs function represents the absolute value, Score_Cos is the cosine similarity score, Score_GPT is the predicted score of the generative large model ChatGPT, and Toal_Score is the full score of the corresponding test question. The confidence C ranges from [0,1]. The larger the value, the more consistent the scores generated by the two schemes are, and the more reliable the scores are. Then set the threshold T. If the confidence is not lower than the threshold T, calculate the average score under the two schemes and output it. Otherwise, enter the active learning process to correct the score.

[0121] (2) Score Correction ① Active learning dataset construction: For the answered questions that are below the confidence threshold T, human experts are invited to judge and give expert scores. Furthermore, the expert scores are used as the true marks of the questions, together with the answer content representation vector to construct a new dataset.

[0122] ②Model update and iteration: The ChatGPT model is optimized using the active learning dataset to improve its ability to identify and evaluate low-confidence answers, and further determine whether the model accuracy meets the requirements. The model accuracy can be determined by comparing the active learning model predictions with the human expert scores. If not, we will still select test samples with prediction scores below the confidence threshold and put them into the model for further training until the model accuracy meets the requirements.

[0123] The present invention proposes a full-cycle ChatGPT-driven automatic examination paper grading method, which has the following advantages over the traditional examination paper review method: Automated and efficient scoring: This invention applies the ChatGPT large model to multiple key links such as test paper content collection, test paper feature extraction, text similarity comparison, and score generation, thereby realizing the automation of test paper review, greatly improving review efficiency, and reducing labor costs.

[0124] Improved accuracy and reliability: The powerful natural language processing capabilities of the ChatGPT large model can accurately understand the text content and perform semantic analysis, thereby improving the accuracy and reliability of scoring.

[0125] Strong interpretability: The present invention calculates the cosine similarity and ChatGPT's predicted score, and combines the weight factor for weighted averaging to obtain the final score, making the score result more transparent and interpretable, which helps students and teachers understand and accept the score results.

[0126] Overcoming the limitations of traditional methods: The present invention overcomes the problem that traditional test paper marking methods are highly dependent on manual feature extraction or data collection, as well as the shortcomings of self-service scoring systems in interpreting scoring results, thereby achieving more efficient, accurate and explainable test paper marking.

[0127] Embodiment 2 The purpose of this embodiment is to provide an automatic examination paper review system based on generative artificial intelligence, including: The test paper acquisition module is configured to: acquire an image of a paper answer paper, and perform preprocessing to obtain a processed answer image; The content recognition module is configured to: perform content recognition on the processed answer image to obtain the content of the answer image; perform abnormality detection on the content of the answer image through the generative artificial intelligence large model, and correct the content of the answer image after obtaining the detection result to obtain the corrected answer image content; The feature extraction module is configured to: extract features from the content of the corrected answer image to obtain a feature vector of the answer image content; extract key information from the content of the corrected answer image using a generative artificial intelligence large model, and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content; The prediction and scoring module is configured to: calculate the cosine similarity score of the feature vector of the answer image content; at the same time, the feature vector of the answer image content is also predicted and scored through the generative artificial intelligence big model; The confidence assessment module is configured as follows: when there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than a given threshold, the predicted score is output.

[0128] Embodiment 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0129] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0130] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.

[0131] Embodiment 5 The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments. The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0132] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0133] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for automatic examination paper review based on generative artificial intelligence, characterized in that: include: Acquire an image of a paper answer sheet, and perform preprocessing to obtain a processed answer image; Performing content recognition on the processed answer image to obtain the content of the answer image; The generative artificial intelligence model is used to detect abnormalities in the content of the answer image, and after obtaining the detection results, the content of the answer image is corrected to obtain the corrected answer image content; Perform feature extraction on the content of the corrected answer image to obtain a feature vector of the answer image content; use the generative artificial intelligence big model to extract key information from the content of the corrected answer image and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content; The cosine similarity score is calculated for the feature vector of the answer image content; at the same time, the feature vector of the answer image content is also predicted and scored through the generative artificial intelligence model; When there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than the given threshold, the predicted score is output.

2. The method for automatic examination paper review based on generative artificial intelligence as claimed in claim 1, characterized in that: The processed answer image content includes mathematical formulas, text and sub-images; The performing content recognition on the processed answer image to obtain the content of the answer image specifically includes: For the mathematical formulas in the processed answer images, the BPD model is used to identify the mathematical formulas; For the text in the processed answer image, use the SideNet model to perform OCR recognition and convert the text in the answer image into text data; For the sub-images in the processed answer image, the image elements in the test paper image are segmented using the image segmentation method to extract image feature representation; The method of performing abnormal detection on the content of the answer image by using the generative artificial intelligence large model, and correcting the content of the answer image after obtaining the detection result to obtain the corrected answer image content, specifically includes: For the content of the answer image, use the confidence scoring technology provided by the OCR software to identify low-confidence text; For texts with low confidence levels that are recognized, the ChatGPT large model is used to detect abnormal words or phrases in the OCR output, and further manual review or automatic correction is performed.

3. The method for automatic examination paper review based on generative artificial intelligence as claimed in claim 1, characterized in that: The corrected answer image content is divided into text and non-text elements; The feature extraction is performed on the corrected answer image content to obtain a feature vector of the answer image content. For the text content, the Word2Vec model is specifically used to embed the text, capture the semantic relationship features between text words, and convert them into a vector representation of fixed dimension; then the pre-trained BERT model is used to sentence encode the text of the answer image to obtain sentence-level semantic relationship features, and convert them into a vector of fixed dimension; For non-text content, a pre-trained image recognition model ResNet is used to extract visual features.

4. The method for automatic examination paper review based on generative artificial intelligence as claimed in claim 1, characterized in that: The method of using the generative artificial intelligence big model to extract key information from the content of the corrected answer image and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content specifically includes: Reconstructing the content of the answer image as input, converting the content of the answer image into a more structured format, and obtaining the reconstructed content of the answer image; The content of the reconstructed answer image is input into ChatGPT, and the semantic information of the key information is extracted through ChatGPT. The semantic information is embedded into the feature vector of the answer image content to form a new feature vector of the answer image content.

5. The method for automatic examination paper review based on generative artificial intelligence as claimed in claim 1, characterized in that: The feature vector of the answer image content includes: a standard answer text vector and an answer text vector; The cosine similarity score is calculated for the feature vector of the answer image content, specifically, the cosine similarity between the standard answer text vector and the answer text vector is calculated to obtain a score prediction based on text similarity; The feature vector of the answer image content is also predicted and scored through the generative artificial intelligence model. Specifically, the standard answer text vector and the answer text vector are communicated using the API interface provided by ChatGPT and sent to the ChatGPT server via an HTTPS request to generate a prediction score result.

6. The method for automatic examination paper review based on generative artificial intelligence as claimed in claim 1, characterized in that: When there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the examinee's answer through the query method of uncertainty sampling, which is specifically expressed as follows: Among them, the Abs function represents the absolute value, Score_Cos is the cosine similarity score, Score_GPT is the predicted score of the generative artificial intelligence model ChatGPT, and Toal_Score is the full score of the corresponding test question; the confidence C value range is [0,1]; Then set a threshold T. If the confidence level is not lower than the threshold T, calculate the average score of the two prediction scores and output it. Otherwise, enter the active learning process to correct the score.

7. An automatic examination paper review system based on generative artificial intelligence, characterized in that: include: The test paper acquisition module is configured to: acquire an image of a paper answer paper, and perform preprocessing to obtain a processed answer image; The content recognition module is configured to: perform content recognition on the processed answer image to obtain the content of the answer image; perform abnormality detection on the content of the answer image through the generative artificial intelligence large model, and correct the content of the answer image after obtaining the detection result to obtain the corrected answer image content; The feature extraction module is configured to: extract features from the content of the corrected answer image to obtain a feature vector of the answer image content; extract key information from the content of the corrected answer image using a generative artificial intelligence large model, and embed it into the feature vector of the answer image content to form a new feature vector of the answer image content; The prediction and scoring module is configured to: calculate the cosine similarity score of the feature vector of the answer image content; at the same time, the feature vector of the answer image content is also predicted and scored through the generative artificial intelligence big model; The confidence assessment module is configured as follows: when there is a serious gap between the cosine similarity score and the predicted score of the generative artificial intelligence large model, a collaborative active learning mechanism is introduced to evaluate the confidence of the candidate's answer through the query method of uncertainty sampling; when the confidence is higher than a given threshold, the predicted score is output.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images