A method and apparatus for correcting a job

Abstract

The application provides a homework correction method and device, wherein the homework correction method comprises the following steps: obtaining a homework to be corrected and a first correction result sequence corresponding to the homework to be corrected; inputting the homework to be corrected and the first correction result sequence into a target homework correction model to obtain a second correction result sequence, wherein the target homework correction model is used to generate a second correction result corresponding to each correction position in the first correction result sequence in parallel; and verifying the first correction result sequence according to the second correction result sequence to obtain a target correction result sequence. While ensuring the accuracy of homework correction, the homework correction efficiency is greatly improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for grading assignments. This application also relates to an apparatus for grading assignments, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, various AI models have emerged. In the education field, AI models are widely used. For example, they can be used for homework grading. In practical applications, users can upload images of their answers to the education system, which can then use the AI ​​model to grade the images and obtain the results.

[0003] However, homework grading in educational settings is characterized by high concurrency. For example, a large number of grading requests are generated at key moments such as after exams or when homework is submitted online. Furthermore, each answer image often contains a significant amount of content that needs grading, thus placing high demands on the grading efficiency of AI models. Therefore, improving the homework grading efficiency of AI models is a crucial issue. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method for job grading. This application also relates to an apparatus for job grading, a computing device, a computer-readable storage medium, and a computer program product, to solve the aforementioned problems existing in the prior art.

[0005] According to a first aspect of the embodiments of this application, a method for grading assignments is provided, comprising: Obtain the jobs to be graded, and obtain the first batch of grading result sequence corresponding to the jobs to be graded; The task to be graded and the first batch of graded result sequence are input into the target task grading model to obtain the second graded result sequence. The target task grading model is used to generate the second graded result corresponding to each graded position in the first batch of graded result sequence in parallel. The first batch of revised results sequence is verified based on the second batch revision result sequence to obtain the target batch revision result sequence.

[0006] According to a second aspect of the embodiments of this application, an apparatus for job grading is provided, comprising: The acquisition module is configured to acquire jobs to be graded and acquire the first batch of grading result sequence corresponding to the jobs to be graded; The input module is configured to input the job to be graded and the first batch of graded result sequence into the target job grading model to obtain the second graded result sequence, wherein the target job grading model is used to generate the second graded result corresponding to each graded position in the first batch of graded result sequence in parallel; The verification module is configured to verify the first batch of correction result sequence based on the second batch correction result sequence to obtain the target batch correction result sequence.

[0007] According to a third aspect of the embodiments of this application, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above-described job grading method.

[0008] According to a fourth aspect of the present application, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described job grading method.

[0009] According to a fifth aspect of the present application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described job correction method.

[0010] The method for grading assignments provided in this application can obtain assignments to be graded and obtain the first batch of grading result sequences corresponding to the assignments to be graded; input the assignments to be graded and the first batch of grading result sequences into a target assignment grading model to obtain a second batch of grading result sequences, wherein the target assignment grading model is used to generate second grading results corresponding to each grading position in the first batch of grading result sequences in parallel; verify the first batch of grading result sequences according to the second batch of grading result sequences to obtain a target grading result sequence.

[0011] One embodiment of this application inputs the first batch of correction result sequences corresponding to the jobs to be corrected into the target job correction model. The target job correction model can then generate second correction results in parallel for each correction position corresponding to the first batch of correction result sequences, thereby obtaining a second batch of correction result sequences. Since the first batch of correction result sequences has already initially provided the first batch of correction results for each correction position, the target job correction model does not need to perform multiple forward propagations to generate each second batch of correction results. Instead, it generates the second batch of correction results based on each correction position in the first batch of correction result sequences, thus improving job correction efficiency. Furthermore, the target correction result sequence can be determined through the second batch of correction result sequences and the first batch of correction result sequences, which can improve the accuracy of job correction. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a method for grading assignments according to an embodiment of this application; Figure 2This is a flowchart of a training method for a homework grading model provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a job correction device provided in one embodiment of this application; Figure 4 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0013] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0014] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0016] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0017] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0018] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0019] First, the terms and concepts involved in one or more embodiments of this application will be explained.

[0020] Large Language Models (LLMs) are deep learning models trained on large amounts of text data, possessing the ability to generate natural language text and understand and generate language. During training, LLMs learn the syntax, semantics, and contextual information of a language to perform various language tasks, such as text generation, text classification, translation, question answering systems, and dialogue generation. Modern LLMs, such as GPT (Generative Pre-trained Transformer), have powerful text understanding and generation capabilities and are widely used in the field of Natural Language Processing (NLP).

[0021] Supervised Fine-Tuning (SFT): SFT is a process of supervising the training of an existing pre-trained model using manually labeled data, so that the model can be adapted to a specific task.

[0022] Optical Character Recognition (OCR): OCR technology is a technique that automatically converts text in images (such as scanned documents, photos, PDFs, etc.) into editable and searchable text data. It uses electronic devices, such as scanners or digital cameras, to capture characters in an image, and then uses pattern recognition technology to translate the shapes of these characters into editable and searchable text data.

[0023] Speculative Decoding is a technical framework for accelerating inference in autoregressive language models. It primarily addresses the slow speed of generating tokens one by one in autoregressive generation. Its core idea is to first use a lightweight draft model to generate a sequence of candidate tokens, and then have the target model validate each candidate token in parallel in a single forward pass, accepting correct tokens and re-evaluating incorrect ones. Specifically, a lightweight draft generator can quickly provide a sequence of candidate tokens, and the target model can then validate all candidate tokens in parallel during a single forward pass, accepting correct tokens and resampling from the first incorrect position, thus reducing the number of forward passes for the target model without sacrificing output quality.

[0024] Draft Sequence: In the speculative decoding process, a sequence of candidate tokens for the candidate outputs is provided in advance by the draft generator. In one or more embodiments of this specification, the first batch of this result sequence can be a draft sequence.

[0025] High-concurrency grading scenarios: These are application scenarios where a large number of students submit their answers simultaneously in an examination or assignment system, and the system needs to return grading results within a very short latency. These scenarios have stringent requirements on inference throughput and response latency.

[0026] The rapid development of artificial intelligence (AI) technology has led to AI models playing a crucial role in intelligent homework grading systems within the education sector. In practical applications, students can upload images of their completed answers to the grading system via electronic devices, allowing the system to call upon the grading model to grade the answers. However, as mentioned in the background section, homework grading scenarios involve high concurrency, with peak times occurring after exams and during homework submissions, placing high demands on the system's responsiveness. Furthermore, current homework grading typically utilizes LLM models based on the Transformer architecture's autoregressive decoding mechanism. This autoregressive decoding mechanism inherently suffers from inference latency and concurrency throughput bottlenecks, impacting the efficiency of homework grading.

[0027] Based on this, this application provides a method for grading assignments, which can obtain assignments to be graded and obtain the first batch of grading result sequences corresponding to the assignments to be graded; input the assignments to be graded and the first batch of grading result sequences into a target assignment grading model to obtain a second batch of grading result sequences, wherein the target assignment grading model is used to generate second grading results corresponding to each grading position in the first batch of grading result sequences in parallel; verify the first batch of grading result sequences according to the second batch of grading result sequences to obtain a target grading result sequence.

[0028] The above-described homework grading method inputs the first batch of grading results sequence corresponding to the homework to be graded into the target homework grading model. The target homework grading model can then generate second grading results in parallel for each grading position corresponding to the first batch of grading results sequence, thus obtaining a second grading result sequence. Since the first batch of grading results sequence has already provided the initial grading results for each grading position, the target homework grading model does not need to perform multiple forward propagations to generate each second grading result; instead, it generates the second grading results based on each grading position in the first batch of grading results sequence, improving homework grading efficiency. Furthermore, the target grading result sequence can be determined through the second grading result sequence and the first batch of grading result sequence, which can improve the accuracy of homework grading.

[0029] This application provides a method for grading assignments, and also relates to an apparatus for grading assignments, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0030] Figure 1 A flowchart of a method for grading assignments according to an embodiment of this application is shown, which specifically includes the following steps: Step 102: Obtain the jobs to be corrected, and obtain the first batch of correction result sequence corresponding to the jobs to be corrected.

[0031] It should be noted that the executing entity for the technical solution in this specification can be any computing device with computing capabilities, such as a server or terminal; this specification does not impose specific restrictions. Furthermore, the executing entity for the method of performing job grading can be the same as or different from the executing entity for the subsequent training method of the job grading model, depending on the actual situation.

[0032] The specific form of the assignment to be graded can be an image of the answer, which can be understood as an image containing the user's answer. It can also be text; this instruction manual does not impose any specific restrictions.

[0033] This manual does not specify any limitations on how the jobs to be graded are obtained. In practical applications, users can send jobs to the computing device via electronic devices, or in other words, send a job grading request to the computing device, which includes the jobs to be graded. The computing device can then obtain the jobs based on this request. Alternatively, users can directly answer the questions on the computing device. After completing the answer, the user can submit the job, and the computing device can respond to the user's submission and obtain the jobs to be graded.

[0034] In one or more embodiments of this specification, after obtaining the assignments to be graded, the first batch of grading results corresponding to each question in the assignments to be graded can be obtained. Specifically, obtaining the sequence of the first batch of grading results corresponding to the assignments to be graded includes: Obtain at least one correction location corresponding to the job to be corrected; Set the first batch of revision results corresponding to each revision position as the specified revision result to obtain the first batch of revision result sequence.

[0035] It should be understood that an assignment awaiting grading may contain several questions and a corresponding answer area for each question. Users fill in their answers in the answer area for each question, thus creating the assignment. Therefore, an assignment awaiting grading contains several grading locations, and each grading location can be understood as the answer area for each question.

[0036] It should be noted that this instruction manual does not restrict how the grading locations in the assignment to be graded are obtained. For example, the assignment to be graded can be input into a preset object detection model, which can locate each question or the corresponding answer area in the assignment. Specifically, it can locate the bounding box of each question or the corresponding answer area in the assignment, and thus output at least one grading location for the assignment. When the assignment is an image, the preset object detection model can be a lightweight object detection model such as YOLOv5-Nano or YOLOv8-Nano. When the assignment is text, the preset object detection model can be a text detection model such as EAST (Efficient and Accurate Scene Text detector) or DB (Differentiable Binarization) used to generate bounding boxes for text lines.

[0037] Of course, each correction location in the work to be corrected can also be marked manually.

[0038] It should be noted that each marking position in the assignment to be marked has a sequential order. In practical applications, each marking position can usually be sorted according to the question number and from left to right to obtain the marking position sequence corresponding to the assignment to be marked. Each marking position in the marking position sequence represents the answer area corresponding to each bounding box.

[0039] In practical applications, the grading results can be preset based on the actual situation and needs. For example, the preset grading results may include: correct, incorrect, partially correct, unanswered, and uncertain. That is, the grading result corresponding to each grading position is one of the following: correct, incorrect, partially correct, unanswered, or uncertain.

[0040] Therefore, in one or more embodiments of this specification, when obtaining the first batch of modification result sequence, the first batch of modification result corresponding to each modification position can be set as a specified modification result, thereby obtaining the first batch of modification result sequence corresponding to the job to be modified.

[0041] The specified grading result can be any of the preset grading results, or it can be set based on actual needs. One or more embodiments of this specification can determine the specified grading result based on prior knowledge. Specifically, if prior knowledge indicates that the current task is of low difficulty or the respondent's knowledge level is high, the grading result tends to have a higher accuracy rate, then the specified grading result can be set to "correct". Alternatively, if prior knowledge indicates that the current task is of high difficulty or the respondent's knowledge level is low, the grading result tends to have a higher error rate, then the specified grading result can be set to "incorrect". For example, if there are three grading positions in the task to be graded, and the specified grading result is "correct", then the first grading result sequence can be "correct, correct, correct".

[0042] In one or more embodiments of this specification, the first batch of grading results sequence can also be constructed based on historical grading statistics. For example, based on historical grading statistics, the grading result with the most occurrences for each question in the assignments to be graded can be obtained, and the grading result with the most occurrences for each question can be used as the first batch of grading results for the grading position corresponding to that question, thereby obtaining the first batch of grading results sequence.

[0043] The above method can improve the efficiency of obtaining the first batch of revised results by setting a specified revised result as the first batch of revised result sequence, or by constructing the first batch of revised result sequence based on historical revised statistical data, thereby improving the efficiency of generating the target revised result.

[0044] In one or more embodiments of this specification, the first batch of revised result sequences can also be obtained through a model. Specifically, obtaining the first batch of revised result sequences corresponding to the job to be revised includes: The task to be graded is input into a preset machine learning model to obtain the first batch of graded result sequence corresponding to the task to be graded. The preset machine learning model is used to generate the first batch of graded result corresponding to each graded position in the task to be graded.

[0045] It should be noted that the preset machine learning model can be the aforementioned draft model. This draft model can be configured based on actual needs; for example, it can be a multimodal model or a pre-trained language model. This draft model can be used to generate the grading results corresponding to each answer area in the assignment to be graded. In practical applications, the assignment to be graded can be input into the draft model, which can generate the first batch of grading results corresponding to each grading position in the assignment, thus obtaining the first batch of grading result sequence.

[0046] Step 104: Input the job to be corrected and the first batch of corrected result sequence into the target job correction model to obtain the second correction result sequence, wherein the target job correction model is used to generate the second correction result corresponding to each correction position in the first batch of corrected result sequence in parallel.

[0047] As mentioned earlier, when using a model based on the autoregressive decoding mechanism of the Transformer architecture for homework grading, the autoregressive decoding mechanism of the Transformer architecture inherently has bottlenecks in inference latency and concurrent throughput. In other words, the autoregressive decoding mechanism based on the Transformer architecture needs to perform a forward propagation when generating the grading result corresponding to each grading position during the homework grading process. Therefore, when there are a large number of questions and a large number of answer areas, the latency is significant and the grading efficiency is low.

[0048] Based on this, this specification allows us to first determine the first batch of correction result sequences corresponding to the jobs to be corrected. Then, this first batch of correction result sequences and the jobs to be corrected can be input into the target job correction model. This first batch of correction result sequences provides the target job correction model with information on the quantity and location of correction results. The target job correction model can then generate the correction results corresponding to each correction location in parallel at once, without needing to perform multiple forward propagations, thus improving job correction efficiency. Specifically, autoregressive inference typically generates correction results token-by-token, while the first batch of correction result sequences can provide the first batch of correction results as placeholders, allowing the model to calculate all correction locations simultaneously.

[0049] In practical applications, the assignments to be graded and the first batch of graded result sequences can be concatenated to obtain a concatenated sequence. This concatenated sequence can then be input into the target assignment grading model all at once. The target assignment grading model can then generate the second graded result sequence in parallel for each graded position in the first batch of graded result sequences through a single forward propagation.

[0050] The target job grading model is used to generate the second grading result corresponding to each grading position in the first grading result sequence in parallel. In one or more embodiments of this specification, the target job grading model can be a multimodal model or a pre-trained language model, and its specific selection can be set based on actual needs.

[0051] Of course, in order to improve the accuracy of the target homework grading model, the target homework grading model can also be obtained by training the initial homework grading model based on the sample homework to be graded, the sample grading result sequence, and the label grading result sequence.

[0052] It should be noted that the first batch of correction results sequence can provide the target job correction model with the quantity information and positional prior of the correction results, so that the target job correction model can clearly identify the correction positions that need to be verified and the expected label corresponding to each correction position, i.e., the first batch of correction results. Thus, the target job correction model can generate the second correction result corresponding to each correction position in parallel in one forward propagation, without having to perform multiple serial forward propagations for each correction position.

[0053] By using the above method, the computational dependency between different grading positions can be eliminated, and multi-round iterative reasoning can be transformed into single-round parallel reasoning. This greatly improves the efficiency of homework grading while ensuring the accuracy of the grading results.

[0054] In practical applications, in order to improve the accuracy of the generated second batch of correction results, prompt information can be preset, and then the preset prompt information, the jobs to be corrected, and the first batch of correction result sequence can be input into the target job correction model.

[0055] Specifically, in one or more embodiments of this specification, the job to be graded and the first batch of graded result sequences are input into the target job grading model to obtain a second batch of graded result sequences, including: The preset prompt information, the job to be graded, and the first batch of graded result sequence are input into the target job grading model to obtain the second grading result sequence. The preset prompt information includes grading rules and grading examples.

[0056] In practical applications, grading rules corresponding to each preset grading result can be pre-set based on actual circumstances and needs. For example, grading rules could be set as follows: if the objective question answer matches the reference answer or is within a reasonable range, the grading result is judged as correct; otherwise, it is judged as incorrect. If the subjective question answer is reasonable, logically sound, and reflects positive values, the grading result is judged as correct; otherwise, it is judged as incorrect. If the answer is partially correct, the grading result is judged as partially correct. If there is no written content or the answer is messy, the grading result is judged as unanswered. If the question and reference information are insufficient, the grading result is judged as uncertain. Correspondingly, grading examples can also be set based on actual needs and circumstances. For example, a grading example could be: "The grading result corresponding to assignment A is B."

[0057] It should be noted that, in practical applications, the preset prompt information may also include the output data format, so as to instruct the target job grading model to output the second grading result sequence according to the output data format.

[0058] By providing preset prompts, reference information can be provided for the target assignment grading model to generate a second grading result sequence, thereby improving the accuracy and efficiency of the second grading result generated by the target assignment grading model.

[0059] Furthermore, in one or more embodiments of this specification, the assignment to be graded may include information from multiple dimensions. In practical applications, user-written images or text can be acquired, and then a preset object detection model can be used to perform layout analysis on the user-written images or text to locate the bounding boxes of each answer area to obtain the grading positions. Additionally, OCR technology can be used to extract the text content of each answer area. The extracted text content can also be organized into structured text according to question number or blank space order. Thus, an assignment to be graded containing user-written images or text, at least one grading position, and the text content of each answer area can be obtained. In this case, more additional information can be provided to the target assignment grading model, thereby further improving the accuracy and efficiency of the generated second grading result.

[0060] Step 106: Verify the first batch revision result sequence according to the second batch revision result sequence to obtain the target batch revision result sequence.

[0061] In one or more embodiments of this specification, after obtaining the second batch correction result sequence, the first batch correction result sequence can be verified based on the second batch correction result sequence to obtain the target batch correction result sequence.

[0062] In one or more embodiments of this specification, the first batch of revised results sequence represents the first batch of revised results corresponding to each revision position in the job to be revised, the second batch of revised results sequence represents the second batch of revised results corresponding to each revision position in the job to be revised, and the target batch of revised results sequence represents the target batch of revised results corresponding to each revision position in the job to be revised. The target batch of revised results corresponding to each revision position is determined based on the first batch of revised results and the second batch of revised results corresponding to each revision position.

[0063] In one or more embodiments of this specification, verifying the first batch of correction result sequences based on the second correction result sequence to obtain a target correction result sequence includes: Compare the second batch of modified result sequences with the first batch of modified result sequences; If the first batch of revision results and the second batch of revision results corresponding to the target revision position are the same, the first batch of revision results or the second batch of revision results shall be used as the target revision result corresponding to the target revision position. If the first batch of correction results and the second batch of correction results corresponding to the target correction position are different, the second batch of correction results shall be used as the target correction result corresponding to the target correction position. Based on the target correction results corresponding to each correction position, obtain the target correction result sequence.

[0064] In practical applications, the first batch of modified result sequences can be compared with the second batch of modified result sequences from left to right, one by one, at each modified position. This applies to the i-th modified position.

[0065] If the second revision result corresponding to the i-th revision position in the second revision result sequence is the same as the first revision result corresponding to the i-th revision position in the first revision result sequence, then the first revision result corresponding to the i-th revision position in the first revision result sequence is accepted. That is, the first revision result corresponding to the i-th revision position in the first revision result sequence can be used as the target revision result corresponding to the i-th revision position. Or, it can be said that the second revision result corresponding to the i-th revision position in the second revision result sequence can be used as the target revision result corresponding to the i-th revision position.

[0066] If the second revise result corresponding to the i-th revise position in the second revise result sequence is different from the first revise result corresponding to the i-th revise position in the first revise result sequence, then the first revise result of the i-th revise position and all subsequent revise positions in the first revise result sequence is rejected, and the second revise result of the i-th revise position in the second revise result sequence can be taken as the target revise result corresponding to the i-th revise position.

[0067] In one or more embodiments of this specification, verifying the first batch of correction result sequences based on the second correction result sequence to obtain a target correction result sequence includes: For each correction location, if the first correction result and the second correction result corresponding to that correction location are the same, the first correction result or the second correction result is taken as the target correction result for that correction location. If the first batch of revision results and the second batch of revision results corresponding to the same position are different, the first batch of revision results corresponding to the same position and each subsequent position are re-acquired, and the second batch of revision results corresponding to the same position and each subsequent position are regenerated, so as to obtain the target revision result corresponding to the same position again. Based on the target correction results corresponding to each correction position, obtain the target correction result sequence.

[0068] In practical applications, if the first batch of correction results and the second batch of correction results corresponding to the current correction position are different, the first batch of correction results corresponding to the current correction position and each correction position after the current correction position can be re-acquired. It should be understood that the method of obtaining the first batch of correction results in step 102 is the same as that of obtaining the first batch of correction results in step 102, which will not be elaborated here. However, the correction positions included in the re-acquired first batch of correction results are a part of the first batch of correction results in step 102. For example, the correction positions include 1, 2, and 3. The first batch of correction results is correct, correct, correct, and the second batch of correction results is correct, incorrect, correct. It can be seen that the first batch of correction results for correction position 1 is the same as the second batch of correction results, the first batch of correction results for correction position 2 is different from the second batch of correction results, and correction position 3 is after correction position 2. Therefore, the first batch of correction results corresponding to correction positions 2 and 3 can be re-acquired. The re-acquired first batch of correction results can be correct, correct, or incorrect, incorrect, etc.

[0069] Then, steps 104-106 can be re-executed based on the newly acquired first batch of revised result sequences. That is, the newly acquired first batch of revised result sequences and the jobs to be revised can be input into the target job grading model to re-acquire the second batch of revised result sequences. Furthermore, the newly acquired first batch of revised result sequences can be verified based on the newly acquired second batch of revised result sequences to re-acquire the target grading results corresponding to each grading position.

[0070] In practical applications, for each correction location, the target job correction model can output the probability distribution of each preset correction result corresponding to that correction location. This allows calculation of the probability value (confidence level) of the first batch of correction results corresponding to that correction location in the first batch of correction result sequences. If this probability value is greater than a preset probability threshold, it can be determined that the second correction result in the second batch of correction result sequences is the same as the first batch of correction results in the first batch of correction result sequences. If the probability value is less than or equal to the preset probability threshold, it can be determined that the second correction result in the second batch of correction result sequences is different from the first batch of correction results in the first batch of correction result sequences.

[0071] The above-described method for grading assignments can obtain assignments to be graded and the first batch of grading result sequences corresponding to the assignments to be graded; input the assignments to be graded and the first batch of grading result sequences into a target assignment grading model to obtain a second batch of grading result sequences, wherein the target assignment grading model is used to generate second grading results corresponding to each grading position in the first batch of grading result sequences in parallel; verify the first batch of grading result sequences based on the second batch of grading result sequences to obtain a target grading result sequence.

[0072] The aforementioned method for grading assignments inputs the first batch of grading result sequences corresponding to the assignments to be graded into the target assignment grading model. The target assignment grading model can then generate second grading results in parallel for each grading position corresponding to the first batch of grading result sequences, thus obtaining a second grading result sequence. Since the first batch of grading result sequences has already provided the initial grading results for each grading position, the target assignment grading model does not need to perform multiple forward propagations to generate each second grading result; instead, it generates the second grading results based on each grading position in the first batch of grading result sequences, improving assignment grading efficiency. Furthermore, the target grading result sequence can be determined using the second grading result sequence and the first batch of grading result sequences, which can improve the accuracy of assignment grading.

[0073] In the job grading method provided in this manual, if all the first batch of grading results in the first batch of grading results sequence are correct, the target job grading model only needs one forward propagation to complete the grading of each grading position, meaning that all the first batch of grading results are accepted. If there are errors in the first batch of grading results sequence, the forward propagation of the target job grading model is also less than that of the token-by-token generation method, thus greatly improving the efficiency of job grading.

[0074] In one or more embodiments of this specification, preset grading results can be added to the vocabulary corresponding to the homework grading model, and each preset grading result is a token. Continuing with the aforementioned preset grading results including correct, incorrect, partially correct, unanswered, and uncertain, the vocabulary corresponding to the homework grading model can contain five grading result tokens, including: correct, incorrect, partially correct, unanswered, and uncertain. Furthermore, trainable embedding vectors can be added to the newly added grading result tokens in the embedding layer of the homework grading model, thereby allowing the initial homework grading model to be trained using training data.

[0075] Each preset grading result token corresponds to a unique token identifier (token ID). For example, the grading result token can be: Correct, Incorrect, Partially Correct, No Answer, Uncertain, and the corresponding token IDs can be: New ID_1, New ID_2, New ID_3, New ID_4, and New ID_5, respectively.

[0076] In this way, each preset grading result token can occupy a token position in the grading result sequence, so that the number of tokens in the grading result sequence can be equal to the number of grading positions in the job to be graded, and the logits of the output head of the target job grading model on the preset grading result token directly correspond to the probability distribution of the preset grading result token.

[0077] In other words, by using a customized tokenizer, all five preset grading results can be represented as a single token, thus eliminating the problem of the general tokenizer splitting the preset grading results into multiple sub-words. Furthermore, based on a speculative decoding mechanism, the number of tokens in the draft sequence (the first batch of grading results) can be precisely equal to the number of grading positions in the job to be graded, thereby ensuring the parallelism of grading result generation. And the logits of the job grading model's output header on these five preset grading result tokens directly correspond to the probability distribution of these five preset grading results.

[0078] In one or more embodiments of this specification, a customized tokenizer can be understood as a special vocabulary extension for each preset grading result. Specifically, as mentioned above, each preset grading result is mapped to a single independent token in the vocabulary corresponding to the job grading model. This ensures that each preset grading result occupies only one token position during the decoding stage, so that the parallel verification granularity of speculative decoding is strictly aligned with the number of answer areas in the job to be graded, or the number of grading positions.

[0079] The method described above expands the vocabulary of the homework grading model by customizing the tokenizer, mapping each of the five types of grading result tokens to a unique new token ID in the vocabulary. This ensures that each grading result strictly corresponds to a unique token during the decoding stage. This avoids the situation where the grading result is split into multiple tokens, which would further increase the number of forward propagations and improve homework grading efficiency.

[0080] In one or more embodiments of this specification, the job grading model may include an embedding layer, an encoding layer, and a decoding layer, wherein the decoding layer may further include a decoding subnet and an output header.

[0081] The embedding layer is used to embed the input data to extract its embedding features. The encoding layer, connected to the embedding layer, generates encoded features based on the embedding features. The decoding layer, connected to the encoding layer, generates prediction results based on the encoded features. Specifically, a decoding subnet is connected to the encoding layer to generate decoding features based on the encoded features, and the output head is connected to the decoding subnet to generate prediction results based on the decoded features.

[0082] This specification also provides, in one embodiment, a method for training the aforementioned target assignment grading model. For example... Figure 2 As shown, Figure 2 A flowchart of a training method for a homework grading model provided in one embodiment of this application specifically includes the following steps: Step 202: Obtain the sample assignments to be graded, the sample grading result sequence, and the label grading result sequence.

[0083] It should be noted that the sample assignments to be graded, the sequence of sample grading results, and the sequence of labeled grading results can be obtained through manual annotation or from relevant training databases.

[0084] Step 204: Input the sample assignments to be graded and the sample grading result sequence into the initial assignment grading model, so that the initial assignment grading model generates the predicted grading result corresponding to each grading position in the sample grading result sequence, and obtains the predicted grading result sequence.

[0085] In one or more embodiments of this specification, the sample job to be graded and the sample grading result sequence are input into an initial job grading model. The initial job grading model can generate the predicted grading result corresponding to each grading position in the sample grading result sequence in parallel, thereby obtaining the predicted grading result sequence.

[0086] Step 206: Based on the predicted grading result sequence and the labeled grading result sequence, train the initial homework grading model until the model training stops, and obtain the target homework grading model.

[0087] In one or more embodiments of this specification, the SFT technique can be used to train the initial homework grading model based on the predicted grading result sequence and the labeled grading result sequence. Specifically, in practical applications, the initial homework grading model can be trained based on the predicted grading result sequence and the labeled grading result sequence, and a loss value can be calculated based on a preset loss function. Specifically, the gradient that minimizes the first loss value can be determined, and the model parameters of the initial homework grading model can be adjusted using gradient descent to minimize the difference between the predicted grading result sequence and the labeled grading result sequence, until the model training stopping condition is met, thus obtaining the target homework grading model.

[0088] The preset loss function can be set based on actual needs. For example, it can be the cross-entropy loss function; this manual does not impose specific restrictions.

[0089] The model training stopping condition can be set based on actual needs. For example, it can be that the number of samples to be graded meets a preset threshold, the determined loss value is less than a preset loss threshold, or the number of iterations meets a preset threshold, etc. This manual does not impose specific restrictions.

[0090] By using the training method of the above-mentioned homework correction model, a target homework correction model can be obtained. This target homework correction model has the ability to generate predicted correction results corresponding to each correction position in the input correction result sequence in parallel based on the homework to be corrected and the input correction result sequence, thereby improving the generation efficiency while ensuring the accuracy of the predicted correction results.

[0091] Corresponding to the above method embodiments, this application also provides an embodiment of a work correction apparatus. Figure 3 A schematic diagram of a job correction device according to an embodiment of this application is shown. Figure 3 As shown, the device includes: The acquisition module 302 is configured to acquire jobs to be graded and acquire the first batch of grading result sequence corresponding to the jobs to be graded; The input module 304 is configured to input the job to be corrected and the first batch of correction result sequence into the target job correction model to obtain the second correction result sequence, wherein the target job correction model is used to generate the second correction result corresponding to each correction position in the first batch of correction result sequence in parallel; The verification module 306 is configured to verify the first batch of modification result sequence based on the second batch modification result sequence to obtain the target batch modification result sequence.

[0092] Optionally, the acquisition module 302 is further configured to acquire at least one grading position corresponding to the job to be graded; set the first batch of grading results corresponding to each grading position as the specified grading result, and obtain the first batch of grading result sequence.

[0093] Optionally, the acquisition module 302 is further configured to input the job to be corrected into a preset target detection model to obtain at least one correction position corresponding to the job to be corrected.

[0094] Optionally, the acquisition module 302 is further configured to input the job to be corrected into a preset machine learning model to obtain the first batch of correction results corresponding to the job to be corrected, wherein the preset machine learning model is used to generate the first batch of correction results corresponding to each correction position in the job to be corrected.

[0095] Optionally, the input module 304 is further configured to input preset prompt information, the job to be graded, and the first batch of graded result sequence into the target job grading model to obtain a second batch of graded result sequence, wherein the preset prompt information includes grading rules and grading examples.

[0096] Optionally, the first batch of revised result sequence represents the first batch of revised results corresponding to each revision position in the job to be revised, and the second batch of revised result sequence represents the second batch of revised results corresponding to each revision position in the job to be revised; The verification module 306 is further configured to compare the second correction result sequence and the first correction result sequence; if the first correction result and the second correction result corresponding to the target correction position are the same, the first correction result or the second correction result is taken as the target correction result corresponding to the target correction position; if the first correction result and the second correction result corresponding to the target correction position are different, the second correction result is taken as the target correction result corresponding to the target correction position; and a target correction result sequence is obtained based on the target correction results corresponding to each correction position.

[0097] Optionally, the first batch of revised result sequence represents the first batch of revised results corresponding to each revision position in the job to be revised, and the second batch of revised result sequence represents the second batch of revised results corresponding to each revision position in the job to be revised; The verification module 306 is further configured to sequentially target each correction position. If the first correction result and the second correction result corresponding to that correction position are the same, the first correction result or the second correction result is taken as the target correction result corresponding to that correction position. If the first correction result and the second correction result corresponding to that correction position are different, the first correction result sequence corresponding to that correction position and each correction position after that correction position is re-acquired, and the second correction result sequence corresponding to that correction position and each correction position after that correction position is regenerated to re-acquire the target correction result corresponding to that correction position. Based on the target correction results corresponding to each correction position, a target correction result sequence is obtained.

[0098] Optionally, the device further includes a training module; The training module is configured to acquire sample assignments to be graded, a sequence of sample grading results, and a sequence of labeled grading results; input the sample assignments to be graded and the sequence of sample grading results into an initial assignment grading model, so that the initial assignment grading model generates a predicted grading result corresponding to each grading position in the sequence of sample grading results, thereby obtaining a sequence of predicted grading results; train the initial assignment grading model based on the sequence of predicted grading results and the sequence of labeled grading results until the model training stops, thereby obtaining a target assignment grading model.

[0099] The aforementioned job grading device can acquire jobs to be graded and acquire the first batch of grading result sequences corresponding to the jobs to be graded; input the jobs to be graded and the first batch of grading result sequences into a target job grading model to obtain a second batch of grading result sequences, wherein the target job grading model is used to generate second grading results corresponding to each grading position in the first batch of grading result sequences in parallel; verify the first batch of grading result sequences according to the second batch of grading result sequences to obtain a target grading result sequence.

[0100] The aforementioned job grading device inputs the first batch of grading result sequences corresponding to the jobs to be graded into the target job grading model. The target job grading model can then generate second grading results in parallel for each grading position corresponding to the first batch of grading result sequences, thus obtaining a second grading result sequence. Since the first batch of grading result sequences has already provided the initial grading results for each grading position, the target job grading model does not need to perform multiple forward propagations to generate each second grading result; instead, it generates the second grading results based on each grading position in the first batch of grading result sequences, improving job grading efficiency. Furthermore, the target grading result sequence can be determined using the second grading result sequence and the first batch of grading result sequences, which can improve the accuracy of job grading.

[0101] The above is a schematic diagram of a job correction device according to this embodiment. It should be noted that the technical solution of this job correction device and the technical solution of the job correction method described above belong to the same concept. For details not described in detail in the technical solution of the job correction device, please refer to the description of the technical solution of the job correction method described above.

[0102] Figure 4 A structural block diagram of a computing device 400 according to an embodiment of this application is shown. The components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and a database 450 is used to store data.

[0103] The computing device 400 also includes an access device 440, which enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 440 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0104] In one embodiment of this application, the aforementioned components of the computing device 400 and Figure 4 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 4 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0105] The computing device 400 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 400 can also be a mobile or stationary server.

[0106] The processor 420 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described job correction method.

[0107] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above-described job grading method belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-described job grading method.

[0108] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described job correction method.

[0109] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above-described job grading method belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-described job grading method.

[0110] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described job correction method.

[0111] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above-described homework correction method belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-described homework correction method.

[0112] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0114] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0115] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0116] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A method of correcting a job, characterized by, include: Obtain the jobs to be graded, and obtain the first batch of grading result sequence corresponding to the jobs to be graded; The task to be graded and the first batch of graded result sequence are input into the target task grading model to obtain the second graded result sequence. The target task grading model is used to generate the second graded result corresponding to each graded position in the first batch of graded result sequence in parallel. The first batch of revised results sequence is verified based on the second batch revision result sequence to obtain the target batch revision result sequence.

2. The method as described in claim 1, characterized in that, Obtain the first batch of revision results sequence corresponding to the job to be revised, including: Obtain at least one correction location corresponding to the job to be corrected; Set the first batch of revision results corresponding to each revision position as the specified revision result to obtain the first batch of revision result sequence.

3. The method as described in claim 2, characterized in that, Obtaining at least one grading location corresponding to the job to be graded includes: The task to be corrected is input into a preset target detection model to obtain at least one correction location corresponding to the task to be corrected.

4. The method as described in claim 1, characterized in that, Obtain the first batch of revision results sequence corresponding to the job to be revised, including: The task to be graded is input into a preset machine learning model to obtain the first batch of graded result sequence corresponding to the task to be graded. The preset machine learning model is used to generate the first batch of graded result corresponding to each graded position in the task to be graded.

5. The method as described in claim 1, characterized in that, Input the jobs to be graded and the first batch of graded result sequences into the target job grading model to obtain the second batch of graded result sequences, including: The preset prompt information, the job to be graded, and the first batch of graded result sequence are input into the target job grading model to obtain the second grading result sequence. The preset prompt information includes grading rules and grading examples.

6. The method according to any one of claims 1 to 5, characterized in that, The first batch of revised results sequence represents the first batch of revised results corresponding to each revision position in the job to be revised, and the second batch of revised results sequence represents the second batch of revised results corresponding to each revision position in the job to be revised; Verify the first batch revision result sequence based on the second batch revision result sequence to obtain the target batch revision result sequence, including: Compare the second batch of modified result sequences with the first batch of modified result sequences; If the first batch of revision results and the second batch of revision results corresponding to the target revision position are the same, the first batch of revision results or the second batch of revision results shall be used as the target revision result corresponding to the target revision position. If the first batch of correction results and the second batch of correction results corresponding to the target correction position are different, the second batch of correction results shall be used as the target correction result corresponding to the target correction position. Based on the target correction results corresponding to each correction position, obtain the target correction result sequence.

7. The method according to any one of claims 1 to 5, characterized in that, The first batch of revised results sequence represents the first batch of revised results corresponding to each revision position in the job to be revised, and the second batch of revised results sequence represents the second batch of revised results corresponding to each revision position in the job to be revised; Verify the first batch revision result sequence based on the second batch revision result sequence to obtain the target batch revision result sequence, including: For each correction location, if the first correction result and the second correction result corresponding to that correction location are the same, the first correction result or the second correction result is taken as the target correction result for that correction location. If the first batch of revision results and the second batch of revision results corresponding to the same position are different, the first batch of revision results corresponding to the same position and each subsequent position are re-acquired, and the second batch of revision results corresponding to the same position and each subsequent position are regenerated, so as to obtain the target revision result corresponding to the same position again. Based on the target correction results corresponding to each correction position, obtain the target correction result sequence.

8. The method according to any one of claims 1 to 5, characterized in that, The target assignment grading model is trained using the following method: Obtain the sample assignments to be graded, the sample grading result sequence, and the label grading result sequence; The sample assignments to be graded and the sample grading result sequence are input into the initial assignment grading model so that the initial assignment grading model generates the predicted grading result corresponding to each grading position in the sample grading result sequence, thereby obtaining the predicted grading result sequence. Based on the predicted grading result sequence and the labeled grading result sequence, the initial homework grading model is trained until the model training stops, thereby obtaining the target homework grading model.

9. A device for grading homework, characterized in that, include: The acquisition module is configured to acquire jobs to be graded and acquire the first batch of grading result sequence corresponding to the jobs to be graded; The input module is configured to input the job to be graded and the first batch of graded result sequence into the target job grading model to obtain the second graded result sequence, wherein the target job grading model is used to generate the second graded result corresponding to each graded position in the first batch of graded result sequence in parallel; The verification module is configured to verify the first batch of correction result sequence based on the second batch correction result sequence to obtain the target batch correction result sequence.

10. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

Images