Intelligent paper marking method based on large language model and optical character recognition

Abstract

The invention relates to the technical field of auxiliary paper marking, and particularly provides an intelligent paper marking method based on a large language model and optical character recognition. The method comprises the steps that test paper and answers are processed through a large language model, test question analysis texts are obtained, and the test question analysis texts comprise analysis texts of the answers; processing the test paper image needing to be marked through an optical character recognition technology to obtain a corresponding answer text; and through comparison of the test question analysis text and the answer text, score data of each question is determined, and the score data comprises scores and comments. According to the intelligent paper marking method provided by the invention, the problem of low paper marking accuracy of an intelligent paper marking method in related technologies can be solved.

Description

Technical Field

[0001] The invention relates to the technical field of auxiliary examination paper marking, and in particular to an intelligent examination paper marking method based on a large language model and optical character recognition. Background Art

[0002] For the test marking task that has long been a heavy burden on teachers, the use of intelligent marking methods to assist in marking has significant advantages over relying solely on manual scoring methods: it can greatly improve the efficiency of test marking; it can conduct in-depth analysis of students' answers and generate a score based on content quality for each answer, avoiding the subjectivity caused by manual review, and is also more explainable; it can also expand the dimensions of scoring.

[0003] However, the current intelligent marking methods still face great challenges: test papers usually contain various handwritings and printed texts, and may also include some special symbols or graphics and other elements, which poses a challenge to accurate character recognition technology. The use of intelligent marking methods requires understanding and parsing a large amount of text content, and will use a large number of tokens (the smallest unit of meaning that the model can understand and generate), which increases the user's usage cost. The use of intelligent marking methods requires understanding the teacher's personalized teaching style during the learning process and making appropriate adjustments when marking test papers. The intelligent marking methods used are required to have strong learning and adaptability. The use of intelligent marking methods requires loading equipment. In order to facilitate the use of teacher groups, the intelligent marking methods used must be applicable to a variety of devices and have good stability.

[0004] Currently, no effective solution has been proposed to address the problems that the intelligent marking methods in related technologies have low recognition accuracy and fail to take into account low cost of use, strong learning and adaptability, ease of use and good stability. Summary of the invention

[0005] The embodiments of the present invention provide an intelligent examination paper marking method based on a large language model and optical character recognition and a computer program product, which at least solve the problem of low recognition accuracy of the intelligent examination paper marking method in the related art.

[0006] An embodiment of the present invention provides an intelligent marking method based on a large language model and optical character recognition, comprising: processing a test paper and an answer through a large language model to obtain a test question analysis text, wherein the test question analysis text includes an analysis text of the answer; processing a test paper image to be marked through optical character recognition (OCR) technology to obtain a corresponding answer text; and determining score data for each question by comparing the test question analysis text with the answer text, wherein the score data includes a score and comments.

[0007] The intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention processes the test paper and answers through the large language model to obtain a test question analysis text, including: performing semantic vectorization processing on the test questions of the test paper to obtain corresponding semantic vectors; based on the semantic vectors, using a similarity matching algorithm to search for relevant materials whose similarity meets preset requirements in a temporary memory library, wherein the temporary memory library is used to cache the knowledge materials involved in the test paper; based on the relevant materials, using prompt word technology to generate corresponding prompt words; inputting the prompt words into the large language model, and the large language model outputs corresponding analysis content.

[0008] The intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention generates corresponding prompt words according to the relevant materials using prompt word technology, including: generating solution steps with chain logic and asking questions through a chain of thoughts according to the questions and answers in the relevant materials; generating corresponding logical self-consistent constraints through a self-consistency verification technology; using the solution steps as prompt examples, wherein the prompt examples are used to input a large language model in a small number of sample prompts to guide the large language model to answer; generating the prompt words according to the questions, the prompt examples and the logical self-consistent constraints.

[0009] The intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention, before searching for corresponding related materials in a temporary memory bank according to the semantic vector, the method also includes: using the test question text of the test paper and the answer text of the answer as a first memory bank; extracting the keyword cluster of the test paper according to the test question text and the answer text; based on the keyword cluster, obtaining relevant content text through the large language model and using it as a second memory bank; building a temporary memory bank of the test paper based on the first memory bank and the second memory bank; and / or, the method also includes: responding to the input operation of the target keyword, querying the relevant content of the target keyword in the temporary memory bank; displaying the relevant content in a preset window.

[0010] The intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention processes the test paper image that needs to be marked through optical character recognition (OCR) technology to obtain the corresponding answer text, including: obtaining a detection frame of the test paper image through the optical character recognition (OCR) technology; from a horizontal perspective, adaptively expanding the detection frame according to the size of the test paper to obtain a first detection frame with complete text lines; from a vertical perspective, using a non-maximum suppression algorithm to filter the redundant first detection frame to obtain a second detection frame whose overlap does not exceed an overlap threshold; based on the second detection frame, using the optical character recognition (OCR) technology to recognize text characters in the second detection frame to obtain the answer text.

[0011] In the intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention, the test paper and the answer are in text format or image format; when the test paper and the answer are in image format, before the test paper and the answer are processed by the large language model to obtain the test question analysis text, the method also includes: processing the test paper and the answer in image format by optical character recognition (OCR) technology to obtain the corresponding answer text.

[0012] The intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention determines the scoring data of each question by comparing the question analysis text with the answer text, including: inputting the question analysis text and the answer text into a large language model, and the large language model outputs corresponding initial scoring data; inputting the initial scoring data into a personality mapping model, and the personality mapping model outputs scoring data with a target language style, wherein the personality mapping model is a machine learning model including multiple fully connected layers, which is obtained by training with multiple groups of training data, and each group of training data includes the initial score of the input large language model, and the score corrected according to the target language style.

[0013] The intelligent marking method based on a large language model and optical character recognition provided by an embodiment of the present invention, after determining the scoring data of each question by comparing the question analysis text with the answer text, the method also includes: displaying the scoring data; responding to the operation of modifying the scoring data, receiving the modified scoring data, and caching it in combination with the scoring data before modification; using the cached scoring data before and after modification as training data to optimize the personality mapping model; and completing the optimization training of the personality mapping model when the target loss of the personality mapping model reaches a preset loss threshold.

[0014] An embodiment of the present invention provides an intelligent marking method based on a large language model and optical character recognition, and the method also includes: responding to an operation of waking up an intelligent assistant, opening a text input box of the intelligent assistant; responding to a text input for calling a function, parsing the calling text for language instructions to obtain a corresponding calling instruction; based on the calling instruction, matching the corresponding functional interface, performing parameter matching, and calling the corresponding function; after turning on the corresponding function, obtaining a call to an associated function, and turning on the corresponding associated function according to the setting.

[0015] A computer program product provided by an embodiment of the present invention includes a computer program / instruction. When the computer program / instruction is executed by a processor, any of the intelligent marking methods based on a large language model and optical character recognition is implemented.

[0016] The intelligent marking method based on a large language model and optical character recognition provided by the embodiment of the present invention solves the problem of low recognition accuracy of the intelligent marking method in the related art, and improves the recognition accuracy by using a large language model and optical character recognition technology; reduces missed detection by using the Look Twice algorithm, and further improves the recognition accuracy; reduces Tokens consumption by setting a temporary memory library and using prompt word technology, making the use of the large language model more efficient and low in cost; builds and uses a personality mapping model through training, so that the score finally output by the large language model is adapted to the marking and grading styles of different teachers, and has strong learning and adaptability; the systems or devices that can run based on the above-mentioned intelligent marking method are diverse, and have good usability and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a flowchart of the steps of an intelligent examination paper marking method based on a large language model and optical character recognition in an embodiment of the present invention.

[0019] Figure 2 Schematic diagram of the temporary memory bank and prompt word technology in an embodiment of the present invention.

[0020] Figure 3 is a schematic diagram of the Look Twice algorithm in an embodiment of the present invention.

[0021] Figure 4 Schematic diagram of a personality mapping model in an embodiment of the present invention.

[0022] Figure 5 It is a flowchart of a large language model memory reflection algorithm in an embodiment of the present invention.

[0023] Figure 6 It is a schematic diagram of the architecture of an intelligent examination system in an embodiment of the present invention.

[0024] Figure 7 It is a schematic diagram of a page for analyzing learning situation in an embodiment of the present invention.

[0025] Figure 8 It is a schematic diagram of a page of intelligent growth in an embodiment of the present invention.

[0026] Fig. 9 yes Figure 6 The functional flow chart of the intelligent marking system is shown.

[0027] Fig.10 It is a page schematic diagram of a teacher creating an examination function page in an embodiment of the present invention.

[0028] Fig.11 It is a schematic diagram of a page for uploading a word sample paper in an embodiment of the present invention.

[0029] Fig.12 It is a schematic diagram of a page for uploading student answer sheets in an embodiment of the present invention.

[0030] Fig.13 It is a schematic diagram of a page for preprocessing student answer sheets based on OCR technology in an embodiment of the present invention.

[0031] Fig.14 It is a schematic diagram of a page for correcting student answer sheets based on a large language model in an embodiment of the present invention.

[0032] Fig.15 Schematic diagram of an adaptive learning interface for a large language model in an embodiment of the present invention.

[0033] Fig.16 It is a page schematic diagram of an intelligent navigation of an instruction generation algorithm in an embodiment of the present invention.

[0034] Fig.17 It is a language logic diagram of the instruction generation algorithm in an embodiment of the present invention.

[0035] Fig.18 Schematic diagram of a knowledge point analysis interface in an embodiment of the present invention.

[0036] Fig.19 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0038] The current intelligent marking method faces great challenges: test papers usually contain various handwritings and printed texts, and may also include some special symbols or graphics, which poses a challenge to accurate character recognition technology. The use of intelligent marking methods requires understanding and parsing a large amount of text content, and will use a large number of tokens, which increases the user's usage cost. The use of intelligent marking methods requires understanding the teacher's personalized teaching style during the learning process and making appropriate adjustments when marking test papers. The intelligent marking method used must have strong learning and adaptability. The use of intelligent marking methods requires loading equipment. In order to be easy for teachers to use, the intelligent marking method used must be applicable to a variety of devices and have good stability.

[0039] To this end, an embodiment of the present invention provides an intelligent examination paper marking method based on a large language model and optical character recognition.

[0040] Please refer to Figure 1 As shown in the flowchart, the embodiment of the present invention provides an intelligent examination paper marking method based on a large language model and optical character recognition, including:

[0041] Step S101, the test paper and the answer are processed by a large language model to obtain a test question analysis text, wherein the test question analysis text includes an analysis text of the answer.

[0042] Step S102, processing the test paper image to be marked by optical character recognition technology to obtain the corresponding answer text.

[0043] Step S103, determining the scoring data of each question by comparing the question analysis text with the answer text, wherein the scoring data includes the score and comments.

[0044] A large language model, also known as a large language model (LLM), is a natural language processing model based on deep learning. It can learn the grammar and semantics of natural language and generate human-readable text. LLM is usually based on a neural network model and trained using a large corpus, such as massive text data on the Internet.

[0045] Optical Character Recognition (OCR) technology can convert text in an image into editable electronic text, thereby enabling computers to recognize and understand text. Optical character recognition usually includes two steps: text detection and text recognition.

[0046] Text detection aims to accurately locate the text content in the input image. In order to achieve this goal, the embodiment of the present invention selects DB Net, a commonly used and efficient text detection baseline. The core technology of DB Net includes Differentiable Binarization (DB for short), which can make the processing of text detection problems with higher complexity (such as rotated or tilted text) relatively simple and efficient. It can process both standard rectangular bounding box text and arbitrary shape text in complex scenes, thereby improving the flexibility and adaptability of text detection. In practical applications, DBNet is mainly used to solve scene text detection problems, and has the advantages of high real-time performance and accuracy.

[0047] Text recognition aims to further identify text areas after locating them through text box detection, and systematically convert these image areas into electronic texts that can be read and understood by machines. At present, text recognition methods based on deep learning mainly include recognition methods based on CTC (Connectionist Temporal Classification) and recognition methods based on the Attention mechanism. The embodiment of the present invention mainly uses the SVTR model, which is mainly based on the Transformer architecture and adopts the Attention mechanism to process relevant information in the input sequence. The SVTR model combines the latest advances in deep learning, can effectively recognize various writing styles and complex text forms, and provide highly accurate recognition results.

[0048] Therefore, the intelligent marking method based on a large language model and optical character recognition provided by the embodiment of the present invention can significantly improve the accuracy of intelligent marking.

[0049] Preferably, in step S101, the test paper and the answers are processed by a large language model to obtain a test question analysis text, including:

[0050] Step S1011, semantic vectorization is performed on the test questions to obtain corresponding semantic vectors.

[0051] Step S1012, based on the semantic vector, a similarity matching algorithm is used to search for relevant materials whose similarity meets preset requirements in a temporary memory library, wherein the temporary memory library is used to cache the knowledge materials involved in the test paper.

[0052] Step S1013, based on relevant materials, using prompt word technology, generate corresponding prompt words.

[0053] Step S1014: input the prompt word into the large language model, and the large language model outputs the corresponding parsed content.

[0054] Specifically, in step S1011, before semantic vectorization is performed on the test questions to obtain corresponding semantic vectors, the test questions are initially parsed by a pattern matching segmentation method to obtain all the questions in the test paper. The pattern matching technology is used to accurately identify the start and end points of the questions to ensure that each question can be processed independently.

[0055] Semantic vectorization processing refers to converting text content into computable vectors through an embedding model to generate a numerical expression that represents the semantics of the question.

[0056] The similarity matching algorithm refers to the semantic distance between the test questions and the materials in the temporary memory bank, and selects the content with the smallest distance (i.e. the highest similarity) as a reference.

[0057] The temporary memory bank stores various materials related to the test paper content, such as background knowledge, historical knowledge, and commentary analysis related to the test questions or answers.

[0058] The temporary memory bank will be dynamically updated according to the test paper content to ensure that relevant materials with sufficiently high similarity can be provided. In addition, the preset requirements for similarity can be determined according to actual needs.

[0059] It can be understood that, through steps S1011 to S1014, the large language model can be used more efficiently and the consumption of Tokens can be reduced.

[0060] Exemplarily, in step S1012, before searching for corresponding related materials in the temporary memory according to the semantic vector, the intelligent marking method provided by the embodiment of the present invention further includes:

[0061] Use the test question text and the answer text of the test paper as the first memory bank.

[0062] Based on the test question text and answer text, the keyword clusters of the test paper are extracted.

[0063] Based on keyword clusters, relevant content text is obtained through a large language model and used as a second memory bank.

[0064] Based on the first memory bank and the second memory bank, build a temporary memory bank for the test paper.

[0065] And / or, the intelligent marking method further includes:

[0066] In response to an input operation of a target keyword, relevant content of the target keyword is searched in a temporary memory bank.

[0067] Displays related content in a preset window.

[0068] The method for building and querying a temporary memory database is not limited to the above examples. Those skilled in the art may choose to refer to the above methods provided in the embodiments of the present invention according to actual needs, or directly use the existing memory database building and query methods.

[0069] Preferably, see Figure 2 As shown, in step S1013, according to relevant materials, using prompt word technology, corresponding prompt words are generated, including:

[0070] Based on the questions and answers in the relevant materials, generate solution steps with chain logic and ask questions through the method of thinking chain.

[0071] Through the self-consistency checking technology, the corresponding logical self-consistent constraints are generated.

[0072] The answering steps are used as prompting instances, wherein the prompting instances are used to input the large language model in the form of a few-sample prompts to guide the large language model to answer.

[0073] Generate prompt words based on the question, prompt examples and logical self-consistency constraints.

[0074] Among them, Chain of Thought (CoT) is a prompt strategy for chain thinking reasoning, which aims to guide the large model to reason step by step and provide a more detailed and logically clear solution process. In an embodiment of the present invention, the solution to each question can be decomposed into multiple steps through the chain of thought, and each step has a logical basis and reasoning process. This can help the large language model output a more logical answer with a clear reasoning chain, rather than just the final result.

[0075] Self-consistency checking technology verifies whether the answer is self-consistent by generating and comparing the output of the large language model multiple times. For example, the same or different prompt words are input into the large language model multiple times to generate multiple versions of parsed answers, and then these parsed answers are cross-compared to ensure that they are consistent in logic and content, thereby improving the reliability of the content generated by the large language model.

[0076] Through the Few-shot Prompt technology, the solution steps are used as prompt examples. Only a small number of examples (such as 1-5 high-quality examples) are needed to guide the large language model to solve the target task. These examples can be pre-prepared standard answers, parsing templates, or relevant materials extracted from a temporary memory bank. In other words, through the prompts of a small number of samples, the large language model can learn the parsing mode and problem-solving logic of these samples, so that it can give more accurate results when solving new questions, and is particularly accurate in specific fields or question types.

[0077] Therefore, the process of generating prompt words is as follows: use the thought chain technology to guide the large language model to gradually reason and solve the problem, and at the same time provide the large language model with reference answer results or ideas through the few-sample prompt technology, and finally use the self-consistency test technology to ensure the consistency and reliability of content generation.

[0078] By inputting the prompt words generated based on the above process into the large language model, high-quality analysis can be generated. These analyses can be directly used for explaining test questions and correcting test papers, and by comparing and adjusting with relevant materials stored in the temporary memory bank, the accuracy of generated content and the efficiency of correction can be further improved.

[0079] Preferably, in step S102, the image of the test paper to be graded is processed by optical character recognition technology to obtain the corresponding answer text, including:

[0080] Step S1021, obtaining the detection frame of the test paper image through optical character recognition technology.

[0081] Step S1022, starting from the horizontal perspective, the detection frame is adaptively expanded according to the size of the test paper to obtain a first detection frame with a complete text line.

[0082] Step S1023, starting from the vertical perspective, using a non-maximum suppression algorithm, filters the redundant first detection frames to obtain a second detection frame whose overlap does not exceed an overlap threshold.

[0083] Step S1024, based on the second detection frame, optical character recognition technology is used to recognize the text characters in the second detection frame to obtain the answer text.

[0084] The Non-Maximum Suppression (NMS) algorithm can eliminate redundant detection boxes (in target detection) or edge points (in edge detection) and retain the parts that are most likely to be real targets or edges.

[0085] Processing the detection frame based on the above steps can overcome the shortcomings of existing OCR technologies, such as Paddle OCR technology, in terms of detection effect. The above steps can improve the overall effect of text recognition by optimizing the accuracy of the detection frame. Figure 3 As shown, the above steps of processing the detection frame can be understood as a Look Twice algorithm for text frame adjustment, which can provide a more reliable text recognition basis for intelligent marking.

[0086] Among them, step S1022 can effectively increase the receptive field and expand the scattered detection frames into a complete text line area. Step S1023 can remove the detection frames with high overlap, which improves the accuracy of the detection frames and reduces interference factors in the text recognition process.

[0087] PaddlePaddle OCR technology mainly includes data preparation, model selection, model training, model evaluation, model optimization, model deployment, text detection, detection box correction, text recognition and other links.

[0088] Data preparation refers to preparing annotated OCR datasets for training and testing the model. The OCR datasets can be existing datasets or self-made datasets.

[0089] Model selection refers to selecting a suitable pre-trained OCR model based on the application scenario and requirements, or you can customize the model according to your needs.

[0090] Model training refers to model training using the selected OCR model and dataset. PaddlePaddle provides technologies such as distributed training and GPU acceleration to improve training efficiency.

[0091] Model evaluation means that after training is completed, the model needs to be evaluated to compare indicators such as model accuracy and operating efficiency.

[0092] Model optimization refers to optimizing the model based on the evaluation results, including adjusting hyperparameters, using more complex network structures, etc.

[0093] Model deployment refers to deploying the trained model into actual applications for text recognition. You can use the corresponding reasoning tools to perform model reasoning and output the results in text format.

[0094] Text detection refers to segmenting the parts of the image to be detected that may contain text to form detection frames.

[0095] Detection frame correction refers to solving the frame offset problem that occurs during text detection to a certain extent through a regression-based correction method.

[0096] Text recognition refers to the process of recognizing text and printing out the text in the detection box.

[0097] It can be understood that the optical character recognition technology adopted in the embodiment of the present invention can refer to the use of optical character recognition technology such as PaddlePaddle. After preliminary detection using optical character recognition technology such as PaddlePaddle, it includes but is not limited to optimizing the detection frame based on the above-mentioned Look Twice algorithm to achieve higher detection accuracy and better text recognition effect than using PaddlePaddle OCR technology.

[0098] Optionally, the test paper and answers are in text format or image format.

[0099] Among them, when the test paper and answers are in image format, in step S101, before the test paper and answers are processed by the large language model to obtain the test question analysis text, the above-mentioned intelligent marking method provided by the embodiment of the present invention also includes: processing the test paper and answers in image format by optical character recognition technology to obtain the corresponding answer text.

[0100] In other words, when the test paper and the answer are in image format, the test paper and the answer are converted into text format so that the large language model can perform subsequent processing. Since the optical character recognition technology used in step S102 can convert images into text, the same optical character recognition technology can be used in the above case.

[0101] Preferably, in step S103, the scoring data of each question is determined by comparing the question analysis text with the answer text, including:

[0102] Step S1031, input the test question analysis text and the answer text into the large language model, and the large language model outputs the corresponding initial scoring data.

[0103] Step S1032, input the initial scoring data into the personality mapping model, and the personality mapping model outputs scoring data with the target language style, wherein the personality mapping model is a machine learning model including multiple fully connected layers, which is trained by multiple sets of training data, each set of training data includes the initial score of the input large language model, and the score corrected according to the target language style.

[0104] It can be understood that the target language style refers to the style reflected in the scores and comments given by different teachers on the test papers they mark based on their own marking experience.

[0105] For example, see Figure 4 As shown, the personality mapping model is a machine learning model including two fully connected layers. The input of the model is the initial rating data, and the output of the model is the rating that is corrected to be close to the teacher's rating style. The initial rating data includes ratings and comments.

[0106] For example, in order to train the above personality mapping model, 200 essays are downloaded from the Internet, and these 200 essays are graded according to the grading standards of key middle school teachers and ordinary middle school teachers respectively, and the teachers' scores are used as labels, so each essay has two labels.

[0107] The scores corrected by teachers of key middle schools and those corrected by teachers of ordinary middle schools were trained and tested respectively. During the training process, for the same essay, the score output by the large language model was input into the personality mapping model, and the personality mapping model output a calibrated score. The mean absolute error loss function (MAE loss) was used for supervision.

[0108] Based on step S1031 and step S1032, the intelligent marking method provided by the embodiment of the present invention has the characteristics of learning and summarizing from the teacher's comments, which can be understood as a large language model memory reflection algorithm, see Figure 5 In other words, the system or device based on the above intelligent marking method can achieve self-growth, and as new constraints and new rules are accumulated and summarized, it becomes closer and closer to the teacher's scoring.

[0109] It is understandable that the new constraints and new rules may be brought about by the above-mentioned prompt word technology provided by the embodiment of the present invention, or may be added manually.

[0110] Exemplarily, in step S103, after determining the scoring data of each question by comparing the question analysis text with the answer text, the intelligent marking method provided by the embodiment of the present invention further includes:

[0111] Display the scoring data.

[0112] In response to the operation of modifying the rating data, the modified rating data is received and cached in combination with the rating data before the modification.

[0113] The cached rating data before and after modification is used as training data to optimize the training of the personality mapping model.

[0114] When the target loss of the personality mapping model reaches a preset loss threshold, the optimization training of the personality mapping model is completed.

[0115] It can be understood that the above method for optimizing the training of the personality mapping model is only a specific example. Those skilled in the art can select other adaptive optimization training methods according to the specific requirements of the system or device used. The system or device is a system or device that operates based on the above intelligent marking method.

[0116] Furthermore, the intelligent marking method provided by the embodiment of the present invention further includes:

[0117] In response to the action of waking up the smart assistant, open the text input box of the smart assistant.

[0118] In response to the input of the calling text of the function, the calling text is parsed for language instructions to obtain the corresponding calling instructions.

[0119] Based on the calling instruction, the corresponding functional interface is matched, and the parameters are matched to call the corresponding function.

[0120] After enabling the corresponding function, obtain the call of the associated function, and enable the corresponding associated function according to the setting.

[0121] It can be understood that, through the above-mentioned response and calling methods, human-computer interaction between the above-mentioned system or device and the user can be realized.

[0122] Based on the above intelligent marking method, the embodiment of the present invention also provides an intelligent marking system, the overall architecture of which can be seen in Figure 6 As shown in the figure, the system is a cross-terminal system built on database, front-end (Vue) and back-end (Django) technologies, supporting mobile phones, PCs and the Gradio application (open source Python library) released by AI Studio (artificial intelligence development and application platform). The system supports users to upload test papers by taking photos and scanning. The perception module of the system uses the above-mentioned OCR technology, and the instruction module is used to convert natural language into functions.

[0123] It is understandable that the intelligent marking method can be used as an intelligent agent and combined with the system through the perception module and the instruction module. The user can communicate with the intelligent agent through natural language to control the system. In other words, the system adopts an intelligent agent-oriented system design.

[0124] The PC terminal is aimed at teachers, and its main functions are examination marking, student status analysis, and intelligent growth. For the student status analysis function, please refer to Figure 7 For the function of intelligent growth, please refer to Figure 8 The mobile terminal is used for displaying learning situation analysis and is aimed at teachers, parents, students and other groups of users.

[0125] Fig. 9 The functional flow chart of the above system is shown, including: uploading word sample papers and preprocessing sample paper knowledge, uploading student answer sheets, preprocessing student answer sheets based on the above OCR technology, and correcting student answer sheets based on the above large language model.

[0126] For the teacher creation test function page, please refer to Fig.10 shown.

[0127] Please refer to the upload word sample Fig.11 As shown, the word sample paper includes test questions and answers. The above system will automatically call the above large language model to process the word sample paper, analyze the knowledge points, and obtain the test question analysis text.

[0128] Please refer to Upload Student Answer Sheets Fig.12 As shown, based on the above OCR technology, the student answer sheets are preprocessed. Fig.13 shown.

[0129] For correcting students' answers based on the above large language model, please see Fig.14 The teacher can conduct a second review on the correction results of the large language model. When the teacher conducts the review, the intelligent experience based on the intelligent marking method will accumulate error samples for the content corrected by the teacher.

[0130] See the adaptive learning interface for the large language model above. Fig.15 As shown, in the process of processing the test paper through the above-mentioned OCR technology, based on the above-mentioned Look Twice algorithm, the above-mentioned temporary memory library, and the above-mentioned prompt word technology, fully automated marking can be achieved accurately and at low cost; based on the above-mentioned personality mapping model, the teacher's review results can be personalized and adapted.

[0131] In order to facilitate teachers to use, the above-mentioned intelligent marking system provided by the embodiment of the present invention adopts an instruction generation algorithm, which can be controlled in a dialogue-based manner throughout the process. For its intelligent navigation page, please refer to Fig.16 This instruction generation algorithm is based on natural language implementation, see Fig.17 As shown, the instruction generation algorithm can be divided into five parts: construction of functional interface, natural language instruction input, natural language instruction parsing, functional interface and parameter matching, data integration and function calling.

[0132] Optionally, the construction of the function interface includes: pre-writing the function functions and interface parameters supported by the above system into an instruction library, which contains all the functions that the above system can perform and their related parameter configurations. The function interface of each function is designed as a standardized function form and connected with specific operation steps or parameters in the database, providing a basis for subsequent instruction generation and calling.

[0133] Optional, natural language command input includes: the user communicates with the built-in AI Bot and inputs a natural language command, which is a function or operation that the user wants to perform, and may include requirements for query, operation, analysis, etc.

[0134] For example: The user can enter: "I want to check the knowledge point analysis of the 2020 College Entrance Examination Chinese simulation paper."

[0135] Among them, AI Bot is the abbreviation of Artificial Intelligence Bot, which is a software program built based on artificial intelligence technology. It can simulate human behavior or intelligence, automatically perform specific tasks or interact with users. It corresponds to the intelligent assistant in the above-mentioned intelligent marking method.

[0136] Optionally, natural language instruction parsing includes: the above-mentioned large language model, as an intelligent analysis module in the above-mentioned instruction module, will perform semantic understanding and parsing of the natural language instructions input by the user. By using NLP (natural language processing) technology, the above-mentioned large language model can convert the core requirements in the natural language instructions (such as viewing test papers, performing analysis, etc.) into standardized commands and identify the functional interfaces related to the user input.

[0137] For example, the large language model above will parse the command "I want to check the knowledge point analysis of the 2020 college entrance examination Chinese simulation paper" to find the function interface to be called and identify the relevant parameters. For example: "2020 college entrance examination Chinese simulation".

[0138] Optionally, the function interface matches the parameters, including: after parsing the core function requirements in the instruction, the above system will find the corresponding function interface by querying the instruction library, and automatically match the required parameters. By connecting the results of natural language parsing with the predefined functions in the instruction library, it is ensured that the correct function is called. In this process, the above large language model not only needs to find the corresponding interface, but also needs to retrieve the ID corresponding to the parameters mentioned in the natural language from the database.

[0139] For example, for the "2020 College Entrance Examination Chinese Simulation", the above system will retrieve its ID value from the database, and the functional function interface corresponding to "View the knowledge point analysis of xxxx" is LLM_preview.

[0140] Optionally, data integration and function calling include: after matching the function interface and parameters, the above system will further perform data integration, integrate multiple input parameters in the user request with the corresponding data, and generate a complete calling instruction that meets the requirements, thereby ensuring that the natural language input by the user can be converted into standardized function calls and realize subsequent functions.

[0141] For example, the above system will call the command LLM_preview(exam_id=5), where exam_id=5 is the ID of the 2020 National College Entrance Examination Chinese Simulation Paper queried from the Structured Query Language (SQL), and then automatically jump to the knowledge point analysis interface, see Fig.18 shown.

[0142] Exemplarily, based on the above intelligent marking system, the embodiment of the present invention also provides experimental data of the above solution, which is as follows:

[0143] Optionally, three handwriting datasets with version numbers CASIA-HWDB2.0 (referred to as HWDB2.0), CASIA-HWDB2.1 (referred to as HWDB2.1), and CASIA-HWDB2.2 (referred to as HWDB2.2) are used as experimental datasets, as shown in Table 1.

[0144] Table 1 Dataset information

[0145] Dataset Number of authors Pages Number of lines Word count HWDB2.0 419 2092 20495 538868 HWDB2.1 300 1500 17292 429553 HWDB2.2 300 1499 14443 380993 total 1019 5091 52230 1349414

[0146] In the HWDB2.0 dataset, there are 419 authors who wrote 2,092 pages of manuscripts, covering 20,495 lines of text, with a total of 538,868 Chinese characters. In the HWDB2.1 dataset, there are 300 authors who wrote 1,500 pages of manuscripts, covering 17,292 lines of text, with a total of 429,553 Chinese characters. In the HWDB2.2 dataset, there are 300 authors who wrote 1,499 pages of manuscripts, covering 14,443 lines of text, with a total of 380,993 Chinese characters.

[0147] The above three data sets have a total of 1,019 authors writing 5,091 pages of manuscripts, covering 52,230 lines of text and a total of 1,349,414 Chinese characters.

[0148] Optionally, the average edit distance of the OCR technology is used as an indicator to experimentally verify the effect of the Look Twice algorithm, as shown in Table 2. The smaller the value of the average edit distance, the higher the recognition rate.

[0149] Table 2 Comparison of experimental results without using the Look Twice algorithm and with using the Look Twice algorithm

[0150] Dataset PaddlePaddle OCR (average edit distance) PaddlePaddle OCR + Look Twice (average edit distance) HWDB2.0 0.3559 0.2446(-0.1153) HWDB2.1 0.3341 0.2548(-0.0757) HWDB2.2 0.3218 0.2233(-0.0985)

[0151] When the OCR technology used in the above-mentioned intelligent marking system is the PaddlePaddle OCR technology, the average editing distance corresponding to the HWDB2.0 data set is 0.3559, the average editing distance corresponding to the HWDB2.1 data set is 0.3341, and the average editing distance corresponding to the HWDB2.2 data set is 0.3218.

[0152] When the OCR technology used in the above-mentioned intelligent marking system is PaddlePaddle OCR + Look Twice technology, the average editing distance corresponding to the HWDB2.0 data set is 0.2446, the average editing distance corresponding to the HWDB2.1 data set is 0.2548, and the average editing distance corresponding to the HWDB2.2 data set is 0.2233.

[0153] It can be found that after using the above-mentioned Look Twice algorithm, the average edit distances corresponding to the three data sets HWDB2.0, HWDB2.1, and HWDB2.2 are reduced by 0.1153, 0.0757, and 0.0985, respectively, indicating that the use of the above-mentioned Look Twice algorithm can greatly improve the recognition effect.

[0154] Optionally, the token utilization rate is used as an indicator to conduct an experimental verification on the AI ​​STUDIO website on the effect of the above prompt word technology. The greater the token utilization rate, the smaller the token consumption is, and the lower the user's usage cost is.

[0155] Compared with the above-mentioned intelligent marking system which does not use the above-mentioned prompt word technology, the above-mentioned intelligent marking system which uses the above-mentioned prompt word technology has a token utilization rate increased by 63%.

[0156] Optionally, 200 high school essays were collected from the Internet, 150 of which were used for training and 50 for testing. The training parameters were set as follows: learning rate 0.05, Epoch (cycle) 100, training set 150, and test set 50, as shown in Table 3.

[0157] Table 3 Training parameter settings

[0158] Training parameters Parameter Value Learning Rate 0.05 Epoch 100 Training set 150 Test Set 50

[0159] The 200 essays were graded according to the grading standards of key middle school teachers and ordinary middle school teachers, and the teachers' scores were used as labels, so each essay has two labels. The scores of key middle school teachers and ordinary middle school teachers were trained and tested separately.

[0160] The mean absolute error (MAE) is used as an indicator to experimentally verify the effect of the personality mapping model, and the experimental results are shown in Table 4. Among them, the smaller the MAE, the closer the score output by the above intelligent marking system is to the teacher's score.

[0161] Table 4 Experimental results

[0162] MAE before fine-tuning MAE after fine-tuning Key middle school teacher 4.98 0.68 Ordinary middle school teacher 5.10 0.67

[0163] Without fine-tuning the personality mapping model, the MAE of the scores obtained by the intelligent marking system and those of key middle school teachers was 4.98, and the MAE of the scores obtained by the intelligent marking system and those of ordinary middle school teachers was 5.10.

[0164] When the above-mentioned personality mapping model is used for fine-tuning, the MAE of the scores obtained by the above-mentioned intelligent marking system and the scores of key middle school teachers is 0.68, and the MAE of the scores obtained by the above-mentioned intelligent marking system and the scores of ordinary middle school teachers is 0.67.

[0165] It can be found that the above-mentioned intelligent marking system using the above-mentioned personality mapping model can output scores that are closer to the teacher's scores.

[0166] An embodiment of the present invention also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, any of the above-mentioned intelligent marking methods based on a large language model and optical character recognition is implemented. Therefore, the above-mentioned computer program product has the same beneficial effects as the above-mentioned intelligent marking method based on a large language model and optical character recognition.

[0167] An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

[0168] refer to Fig.19, a block diagram of an electronic device that can be used as a server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or required herein.

[0169] like Fig.19 As shown, the electronic device includes a computing unit 1901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 to a random access memory (RAM) 1903. In RAM 1903, various programs and data required for the operation of the electronic device can also be stored. The computing unit 1901, ROM 1902, and RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0170] Multiple components in the electronic device are connected to the I / O interface 1905, including: an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 can be any type of device that can input information to the electronic device, and the input unit 1906 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 1907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 can include but is not limited to a disk, an optical disk. The communication unit 1909 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, and / or a wireless communication transceiver, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0171] The computing unit 1901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the method embodiments created by the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 1908. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1902 and / or a communication unit 1909. In some embodiments, the computing unit 1901 may be configured to perform the above method in any other appropriate manner (e.g., by means of firmware).

[0172] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0173] In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0174] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0175] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0176] The various steps described in the method implementation methods provided in the embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0177] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same and similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0178] The above-described embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of protection. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. An intelligent examination paper marking method based on a large language model and optical character recognition, characterized in that: include: Processing the test paper and the answer through a large language model to obtain a test question analysis text, wherein the test question analysis text includes an analysis text of the answer; The test paper images to be graded are processed through optical character recognition (OCR) technology to obtain the corresponding answer text; By comparing the test question analysis text with the answer text, the scoring data of each question is determined, wherein the scoring data includes a score and comments.

2. The method according to claim 1, characterized in that The test papers and answers are processed through a large language model to obtain the test question analysis text, including: Performing semantic vectorization processing on the test questions of the test paper to obtain corresponding semantic vectors; According to the semantic vector, a similarity matching algorithm is used to search for relevant materials whose similarity meets preset requirements in a temporary memory bank, wherein the temporary memory bank is used to cache the knowledge materials involved in the test paper; According to the relevant materials, using prompt word technology, generating corresponding prompt words; The prompt word is input into the large language model, and the large language model outputs the corresponding parsed content.

3. The method according to claim 2, characterized in that According to the relevant materials, corresponding prompt words are generated by using prompt word technology, including: Based on the questions and answers in the relevant materials, generate solution steps with chain logic and ask questions through the method of thinking chain; Generate corresponding logical self-consistent constraints through self-consistency verification technology; The answering step is used as a prompting example, wherein the prompting example is used to input a large language model in a small sample prompting manner to guide the large language model to answer; The prompt word is generated according to the question, the prompt instance and the logical self-consistency constraint.

4. The method according to claim 2, characterized in that: Before searching for corresponding related materials in a temporary memory according to the semantic vector, the method further includes: Based on the test question text of the test paper and the answer text of the answer as the first memory bank; Extracting keyword clusters of the test paper according to the test question text and the answer text; Based on the keyword cluster, the relevant content text is obtained through the large language model and used as a second memory bank; Building a temporary memory bank of the test paper based on the first memory bank and the second memory bank; And / or, the method further comprises: In response to an input operation of a target keyword, searching the temporary memory for content related to the target keyword; The related content is displayed in a preset window.

5. The method according to claim 1, characterized in that The test paper images that need to be graded are processed through optical character recognition (OCR) technology to obtain the corresponding answer text, including: Obtaining a detection frame of the test paper image by using the optical character recognition (OCR) technology; Starting from a horizontal perspective, the detection frame is adaptively expanded according to the size of the test paper to obtain a first detection frame with a complete text line; From a vertical perspective, a non-maximum suppression algorithm is used to filter out redundant first detection frames and obtain a second detection frame whose overlap does not exceed the overlap threshold. Based on the second detection frame, the optical character recognition (OCR) technology is used to recognize the text characters in the second detection frame to obtain the answer text.

6. The method according to claim 5, characterized in that The test paper and answers are in text format or image format; When the test paper and the answer are in image format, before the test paper and the answer are processed by the large language model to obtain the test question analysis text, the method further includes: The test papers and answers in image format are processed through optical character recognition (OCR) technology to obtain the corresponding answer text.

7. The method according to claim 1, characterized in that By comparing the test question analysis text with the answer text, the scoring data of each question is determined, including: Inputting the test question analysis text and the answer text into a large language model, and having the large language model output corresponding initial scoring data; The initial scoring data is input into a personality mapping model, and the personality mapping model outputs scoring data with a target language style, wherein the personality mapping model is a machine learning model including a plurality of fully connected layers, and is obtained by training with a plurality of sets of training data, each set of training data including an initial score of an input large language model, and a score corrected according to the target language style.

8. The method according to claim 7, characterized in that After determining the score data of each question by comparing the question analysis text with the answer text, the method further includes: Displaying the scoring data; In response to the operation of modifying the rating data, receiving the modified rating data and caching it in combination with the rating data before the modification; Using the cached rating data before and after modification as training data, optimizing and training the personality mapping model; When the target loss of the personality mapping model reaches a preset loss threshold, the optimization training of the personality mapping model is completed.

9. The method according to claim 1, characterized in that: The method further comprises: In response to the operation of waking up the intelligent assistant, a text input box of the intelligent assistant is opened; In response to the input of a calling text of a function, the calling text is parsed for a language instruction to obtain a corresponding calling instruction; Based on the calling instruction, the corresponding functional interface is matched, and parameter matching is performed to call the corresponding function; After enabling the corresponding function, obtain the call of the associated function, and enable the corresponding associated function according to the setting.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images