Work automatic correction and trace feedback method and system based on multi-modal large model

Abstract

This invention discloses an automatic homework grading and feedback method and system based on a multimodal large model, belonging to the fields of artificial intelligence and educational informatization. The method first acquires and corrects handwritten homework images, then performs feature analysis on the image content, and adaptively selects either visual understanding grading or text recognition grading path accordingly. If the output confidence score of the text path is lower than a threshold during processing, the visual path will be automatically triggered to review the same content. After grading, the system uses coordinate mapping and image fusion technology to overlay visual grading marks on the original homework image using the parsed score and location information, generating a feedback image that mimics the teacher's handwriting. This invention overcomes the shortcomings of existing multimodal grading systems, such as modal isolation and rigid collaboration, by introducing intelligent routing decision-making and cross-modal review mechanisms, significantly improving the adaptability, reliability, and user experience of grading.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and natural language processing technology, specifically a method and system for automatic homework grading and feedback based on a multimodal large model. Background Technology

[0002] In the field of intelligent education, automatic grading technology combining computer vision and natural language processing has made significant progress. Currently, the industry generally recognizes that a single technical approach has limitations: purely visual methods struggle to understand deep semantics, while purely text recognition (OCR) methods cannot process non-textual information such as graphics and charts.

[0003] The current technical bottleneck of homework grading systems lies not in the number of processing modalities, but in the lack of a collaborative mechanism. Existing solutions mainly suffer from two limiting paths: one is to rely on manually preset rules to statically allocate processing modalities; the other is to use hard-coded logic based on fixed question types for mode selection. Both paths exhibit inherent rigidity and are unable to effectively cope with the high complexity and hybridity of question formats in real-world homework scenarios. For example, a comprehensive question that integrates textual arguments, complex formulas, and diagrams has dynamic and complex processing requirements, which static rules cannot respond to with nuance. Moreover, the visual processing module and text recognition module in the existing architecture are usually in a state of isolation. When the confidence level of the text recognition path is significantly reduced due to factors such as illegible handwriting or complex page layout, the entire system lacks an endogenous, automated perception mechanism to recognize this uncertainty, let alone intelligently trigger process switching based on this state to schedule the visual module to cross-verify the same answer. The isolation between these modalities and the lack of a collaborative feedback mechanism mean that the final correction results of the system lose reliability in key aspects, constituting a core obstacle for existing technologies to move towards practical application and high reliability. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for automatic job grading and feedback based on a multimodal large model, so as to solve the above-mentioned problems.

[0005] The technical solution of this invention is: An automatic homework grading and feedback method based on a multimodal large model includes: Acquire images of students' handwritten assignments to be graded and corresponding reference answer data; preprocess the images of students' handwritten assignments to identify and correct the rotation direction of the images; perform content feature analysis on the corrected images, the content features including at least the proportion of graphic areas and formula complexity, and generate routing decisions based on the analysis results; Based on the routing decision, the system adaptively selects to execute either a visual correction process or a text correction process. The selected correction process is executed as follows: If the visual correction process is selected, the image region is input into a multimodal large model for visual reasoning to obtain a first structured output containing the score and the coordinates of the correction mark in the image; if the text correction process is selected, OCR recognition is performed first to obtain the text, and then the text is input into a processing model for semantic reasoning to obtain a second structured output containing the score and confidence level; if the confidence level of the text correction process output is lower than a preset threshold, the system automatically reverts and triggers the visual correction process to review the same content; the confidence level of the text correction process output is the sole criterion for triggering cross-modal review. When the confidence level is lower than the preset threshold, the system automatically reverts and triggers the visual correction process to review the same content, thus forming a closed loop to ensure the reliability of the correction.

[0006] Parse the first or second structured output to extract the scoring results and the location coordinates of the grading marks; Traceable rendering: Based on the location coordinate information, the corresponding visual correction mark is superimposed on the student's handwritten homework image to generate a traceable correction image, and the traceable correction image and the scoring report are output.

[0007] Furthermore, the routing decision rules include: When the question area is detected to contain geometric figures, function graphs, or charts, it is determined to be a high-complexity visual task, and the routing decision is to execute the visual grading process. When the pixel density of the formula region in the corrected image exceeds the preset complexity threshold, it is determined to be a high-complexity visual task, and the routing decision is to execute the visual correction process. When the question area is detected to consist only of plain text or simple formulas, and no graphic area is detected, it is determined to be a text semantic task, and the routing decision is to execute the text grading process. For ambiguous situations between the two mentioned above, the image is pre-identified using OCR and a confidence level is obtained. If the confidence level is lower than a set threshold, the routing decision is to execute a visual correction process.

[0008] The complexity threshold and confidence threshold can be dynamically adjusted based on the overall quality of the task image or the metadata of the subject.

[0009] The routing decision rules transform abstract content feature analysis into executable, concrete judgment logic. Through the synergy of multiple criteria, including image detection, formula complexity thresholds, and OCR pre-recognition confidence, a multi-layered, executable intelligent decision-making logic is constructed. This upgrades the system's routing decision from a mechanical arrangement of modalities to a dynamic, optimal selection based on the characteristics of the question content. This fundamentally ensures a precise match between the processing path and the question type requirements, and guarantees that the system can adaptively select the best processing path for diverse question types.

[0010] Furthermore, the visual correction process specifically includes: Construct a prompt word for locating the question region. This prompt word contains the following instruction: "Please identify the answer region for each question in the assignment image and return the boundary coordinates of each region." Construct prompts for question location to guide a multimodal visual model to identify the answer area for each question and obtain its coordinate frame; Based on the coordinate frame, the student assignment image is cropped into individual question slice images; Construct a grading prompt word containing the following instructions: "Please analyze the problem-solving steps in the input image, determine whether they are correct or incorrect, return the relative coordinates of the grading mark for each step in the problem slice image, and identify the location coordinates of the final answer area for scoring record keeping." Input the sequence of sliced ​​images of the question and the correction prompts into a multimodal large model; The visual encoder of the multimodal large model is used to perceive the handwritten trajectory and geometric structure, and output a JSON-formatted report containing the score, step correction coordinates and the coordinates of the final answer box.

[0011] By using multimodal large-scale model visual understanding and prompt word engineering to achieve image slicing, combined with specialized prompt words containing instructions for region localization, grading and scoring, and answer box location, the multimodal large-scale model is guided to perform end-to-end visual reasoning, and directly outputs structured grading results with precise location information from the image, providing core technical support for solving complex question types that include graphics and charts.

[0012] Furthermore, when parsing the JSON report containing normalized coordinates, the local coordinates are restored to the full-image pixel coordinates of the student's handwritten homework image using the following formula: Global X = Slice X + Normalized x × Slice width, Global Y = Slice Y + Normalized y × Slice height, thus restoring the local coordinates to the full-image pixel coordinates.

[0013] The coordinate mapping formula described above establishes a deterministic conversion relationship from the local relative coordinates output by the model to the global pixel coordinates of the original job, realizing precise spatial alignment of correction marks and handwritten content, and in-situ feedback of traces.

[0014] Furthermore, the text correction process specifically includes: The OCR engine is invoked to perform text detection and recognition on the entire assignment page to obtain full-text text information; Using a text layout analysis algorithm and combining question number features, the full text information is segmented into independent "question-answer" text pairs; Retrieve the standard answer text corresponding to the current question from the vector database; Construct a second multimodal cue word that includes semantic scoring and logical reasoning instructions; The "question-answer-response" triplet text is input into the second processing model for semantic reasoning to obtain the second structured output.

[0015] By using full-text OCR recognition, question-answer text pair segmentation, and vector database retrieval, a structured input suitable for deep semantic understanding of large language models is constructed, providing a standardized processing paradigm for efficient and accurate grading of plain text and simple arithmetic problems.

[0016] Furthermore, the trace rendering also includes the following steps: Preload a library of correction icon materials in various handwritten styles; The scaling ratio of the correction icon is dynamically calculated based on the size and resolution of the student's handwritten homework image; By using the Alpha channel blending technique of the graphics processing library, the scaled correction icon layer is overlaid onto the corresponding coordinate position of the original homework image layer with a specified transparency, generating a visual composite image that resembles a teacher's handwritten corrections.

[0017] By preloading handwritten-style icons, dynamically scaling, and using alpha channel blending technology, this step not only achieves accurate overlay of correction information but also further simulates the visual texture of teachers' handwritten corrections, significantly improving the intuitiveness of the feedback results.

[0018] Furthermore, it also includes a scoring mechanism, specifically supporting the following two optional scoring modes: Preset score mode: Before the grading begins, the user pre-sets the score for each question and embeds the score information into the first or second multimodal prompt words, so that the large model directly outputs the actual score for each question based on the preset score. Percentage conversion mode: The multimodal large model or processing model outputs an AI score percentage for each question out of a full score of 100. After grading, the user sets the actual full score for each question, and the system automatically calculates the final score according to the formula "actual score = question score × (AI score percentage / 100)".

[0019] The scoring mechanism offers two flexible scoring modes: preset scores and percentage conversion. This allows the system to adapt to the strict grading requirements of fixed scores in standardized tests, as well as to support flexible conversion of score ratios in process-based assessments, thus enhancing the system's practicality in different teaching scenarios.

[0020] A multimodal large-scale model-based intelligent job grading system, used to implement the above-mentioned automatic job grading and feedback method, includes: The image acquisition and preprocessing module is used to acquire images of students' handwritten homework to be graded and corresponding reference answer data, and to perform orientation detection and correction on the images; The adaptive routing decision module is used to perform content feature analysis on the corrected job images and generate routing decisions based on the analysis results to adaptively select the grading process. It includes an image detection unit, a formula analysis unit, and a decision engine. A dual-modal correction engine is used to execute the selected visual correction process or text correction process. It includes a visual correction component and a text correction component that work in parallel. The text correction component integrates a confidence judgment and rollback trigger unit for triggering the visual correction component to perform a review when the confidence level is insufficient. The results synthesis and rendering module is used to parse the output of the grading engine to extract scores and coordinates, and to overlay grading marks on the job image based on the coordinates, generating and outputting a grading image with traces and a grading report.

[0021] Furthermore, the visual correction component integrates a multimodal large model for receiving images and instructions and outputting structured data containing coordinates; the text correction component integrates an OCR engine and a text processing model. The system adopts a distributed architecture that coordinates local terminals and cloud servers.

[0022] The local terminal is responsible for scanning and collecting the images of the students' handwritten assignments, uploading them, and finally receiving, synthesizing, and displaying the images with the grading record. The cloud server is equipped with the adaptive routing decision module and the bimodal grading engine, which receive image data and perform intelligent grading. It returns structured information, including scores and coordinates, to the local terminal via a RESTful API (Representational State Transition Application Programming Interface). This distributed architecture deploys the computationally intensive AI grading service in the cloud, leaving lightweight interaction and rendering tasks locally. This optimizes the allocation of computing resources and ensures high system scalability, while standardized interfaces guarantee efficient and reliable collaboration.

[0023] Compared with the prior art, the beneficial effects of the present invention are: This invention transforms the fragmented and mechanically stacked processing capabilities of traditional multimodal systems into an organic intelligent whole with autonomous decision-making and dynamic collaboration capabilities. Specifically, by analyzing the content features of the task images and implementing adaptive routing, the system achieves accurate and dynamic matching of processing strategies for different question types, especially complex questions that integrate text, formulas, and graphics, completely overcoming the rigidity and limitations caused by relying on preset rules or hard-coded logic. Furthermore, through a built-in confidence-triggered cross-modal review mechanism, an inherent reliability guarantee is established at the system architecture level. When the confidence level of the text recognition path output falls below a preset threshold, this mechanism can autonomously trigger a process switch, scheduling the visual understanding path to perform a secondary analysis of the same answer. This enables the system to dynamically evaluate the reliability of its intermediate results and activate preset fault-tolerant processes when the evaluation fails. Thus, without human intervention, it significantly reduces final grading errors caused by the failure of a single recognition link, improving the robustness of the grading process and the accuracy of the output results from the root of the system design.

[0024] Meanwhile, this invention upgrades the feedback from a text or data report independent of the original assignment to an enhanced image that precisely corresponds spatially to and visually integrates with the original handwritten content. Through coordinate mapping and image synthesis technology, the system directly overlays the grading conclusions (correct or incorrect, score, comments) as visual markers onto the corresponding answer positions in the student's original assignment image, thus achieving an in-situ integrated presentation of grading information and the answer context. This eliminates the cognitive burden of manually comparing and locating the grading results with the original assignment, making the feedback more intuitive and the location more accurate, significantly improving the efficiency of information delivery and the user experience. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the automatic grading method for handwritten homework of the present invention.

[0026] Figure 2 This is a schematic diagram of the module architecture of the intelligent correction system of the present invention.

[0027] Figure 3 This is a diagram of the question area after the system of this invention performs AI intelligent segmentation on university mathematics assignments.

[0028] Figure 4 This is a diagram showing the effect of the present invention in the context of leaving traces for grading university math assignments.

[0029] Figure 5 This is a schematic diagram of the end-to-end interaction architecture between the local terminal and the cloud server of this invention.

[0030] Figure 6 This is a schematic diagram illustrating the core principle logic of the dual-modal correction method of this invention. Detailed implementation manners

[0031] The following combines Figures 1 to 6 to describe the detailed implementation manners of the present invention in detail.

[0032] It should be noted that the circuit connections involved in the present invention all adopt conventional circuit connection methods and do not involve any innovation.

[0033] Embodiment As Figure 1 shown, a method for automatic correction and trace feedback of homework based on a multimodal large model includes the following steps First, the system obtains an image of a student's handwritten homework through a scanner or a document camera, and simultaneously obtains the electronic version of the reference answer data corresponding to the homework. Then, the system preprocesses the obtained homework image and detects that the image has been rotated 90 degrees counterclockwise due to the shooting angle problem, and then automatically corrects it to make the image in the forward reading direction, as Figure 3 shown. Then, the system performs content feature analysis on the corrected image. Through the algorithm, it calculates that the proportion of the geometric figure area in the total answer area in the image is about 40%, and at the same time identifies a complex formula area containing integral signs and fractional structures, and its pixel density exceeds the preset complexity threshold. Based on this analysis result, the system generates a routing decision: determining that this homework belongs to a high-complexity visual task.

[0034] As Figure 6 shown, according to this routing decision, the system adaptively selects to execute the visual correction process. The system inputs the image of the question area containing complex formulas and figures into a pre-trained multimodal large model (such as GPT-4V, Qwen-VL, etc.) for visual reasoning. After analyzing the problem-solving steps, the large model returns a first structured output, which is a JSON-format report. It not only contains the scoring conclusion of "the calculation in step 2 is incorrect, deduct 5 points" for this question, but also indicates the position where the correction mark (such as a cross "×") should be placed in the form of normalized coordinates, for example {"mark": "×", "x": 350, "y": 120}. In another case, if the system analyzes and finds that the homework content is only a pure text ancient poem dictation, it will be routed to the text correction process. This process first performs OCR recognition on the image, converts the handwritten text into text such as "Before my bed a pool of light", and then inputs this text into a large language model for semantic reasoning to obtain a second structured output containing the score and recognition confidence (such as 95%). In particular, if the confidence of this OCR recognition is only 60% (lower than the preset threshold of 75%) due to scribbled handwriting, the system will automatically roll back and trigger the visual correction process to perform a secondary review on this area to ensure accuracy.

[0035] Subsequently, the system analyzes the structured output returned by the large model, extracting the scoring results and coordinate information. Finally, it enters the trace rendering stage. Based on the extracted coordinate information, such as converting normalized coordinates (350, 120) into a specific pixel position (455, 300) in the entire image, the system retrieves a handwritten-style red cross icon from a pre-set material library and precisely overlays it onto the corresponding position on the student's original assignment image using image processing technology. Simultaneously, a score "95 / 100" marker is generated and overlaid on the side of the image. Ultimately, the system outputs a composite image incorporating all the marking traces and a detailed JSON-formatted scoring report, completing the entire marking process.

[0036] In a specific grading task of this system, the adaptive routing decision module executed the following rules to generate a decision: For a function problem that includes a hand-drawn parabolic graph, the graph detection unit successfully identified the graph elements, and the system immediately determined it to be a high-complexity visual task, routing it to the visual grading process.

[0037] For a calculus problem that contains triple integral symbols but no graphics, the formula analysis unit calculates that the pixel density of its formula area is as high as 800 (exceeding the threshold of 500), and it is also judged as a high-complexity visual task, which is then routed to the visual grading process.

[0038] For a question that only requires writing chemical equations, if no graphs are found and the formula is simple (such as H2+O2→H2O), it is determined to be a text semantic task and routed to the text grading process.

[0039] For a chemistry question containing a few hand-drawn diagrams of experimental setups but primarily consisting of textual descriptions, the problem is in a state of ambiguity. The system first performs a rapid OCR pre-recognition, obtaining a confidence level of 70%, which is lower than the 85% threshold set for ambiguity. Therefore, the final decision is still to execute a visual correction process to ensure accurate understanding of the diagrams.

[0040] like Figure 4 As shown in the figure, this example demonstrates the grading results for two typical problems (indefinite integral calculation and matrix eigenvalue solving). The red checkmarks and score markings represent the in-situ grading results generated after understanding the problem-solving logic.

[0041] Taking a physics assignment containing three handwritten answer questions as an example, the visual grading process unfolds as follows: First, the system constructs a prompt for locating the answer area of ​​each question in the assignment image, which reads: "Please identify the answer area of ​​each question in the assignment image and return the boundary coordinates of each area." This prompt, along with the assignment image, is input into a multimodal visual model, which returns the coordinate frames of the answer areas for the three questions. Next, the student's assignment image is cropped into independent question slice images based on the coordinate frames. Then, the system constructs a grading prompt, which reads: "Please analyze the solution steps in the input image, determine their correctness, return the relative coordinates (range 0-1000) of the grading mark (checkmark or cross) for each step in the question slice image, and identify the location coordinates of the final answer area for scoring purposes." Subsequently, the three slice image sequences, along with the grading prompt, are input into a multimodal visual model (such as InternVL) deployed in the cloud. The model's visual encoder deeply perceived the handwritten trajectories, circuit diagram structures, and formula writing in the images, and finally output three JSON reports, each containing the scoring results, step correction coordinates, and answer box trace coordinates.

[0042] Continuing the previous example, the system receives a JSON report for the first question slice (whose top-left corner coordinates in the entire image are (slice X, slice Y) (50, 200), the width slice is 600 pixels wide, and the height slice is 400 pixels high). The report indicates that the normalized coordinates of the erroneous step are (x: 200, y: 500). The parsing module calculates using the formulas: global X coordinate = 50 + (200 / 1000) * 600 = 50 + 120 = 170; global Y coordinate = 200 + (500 / 1000) * 400 = 200 + 200 = 400. Thus, the system accurately restores the local coordinates (200, 500) to the specific pixel coordinates (170, 400) in the entire task image, laying the foundation for subsequent accurate overlay of correction marks.

[0043] For a handwritten English reading comprehension answer sheet, the system initiates a text grading process: First, it uses Baidu OCR or Tesseract engines to perform text detection and recognition on the entire page image, obtaining the text information and location of all English words. Next, using a rule-based text layout analysis algorithm, it identifies question number features such as "Question 1:" and segments the entire text into several independent "question-answer" text pairs. Then, the system quickly matches the standard answer text for the current question from a pre-built vector database through semantic similarity retrieval. Afterward, it constructs a second multimodal prompt, such as "Please compare the following students' answers with the standard answer, score them from three aspects: grammar, vocabulary, and content relevance, and provide a confidence level." Finally, the retrieved standard answer, the student's answer identified by OCR, and the prompt word are combined into a triple and input into a second processing model (such as ChatGPT or Wenxin Yiyan) for semantic reasoning to obtain a structured output, for example: {"score": 8, "confidence": 0.88, "feedback": "Tense usage is basically correct, but there is one spelling error."}.

[0044] During the rendering phase, the system underwent enhancement processing to improve the visual effect. The system preloaded a library of handwritten-style correction icons containing various colors and stroke thicknesses. For the current 2480x3508 pixel resolution assignment image, the system dynamically calculated that the correction icons should be scaled down to 50% of their original size. Subsequently, using alpha channel blending techniques from graphics libraries such as OpenCV or PIL, a semi-transparent (70% opacity) red checkmark icon layer was precisely overlaid onto the original assignment image layer at the calculated global coordinates (170, 400). Through this blending, the final composite image naturally integrates the correction marks with the original handwriting, presenting a realistic visual effect as if a teacher had personally corrected the work with a red pen.

[0045] The system offers flexible scoring modes to adapt to different teaching scenarios. In the preset score mode, teachers set the score for the first question of the assignment to 15 points before grading. This score is embedded in the prompt sent to the large learning model (e.g., "This question is worth 15 points"), and the model directly outputs a structured result of "Score: 12 points". In the percentage conversion mode, if the teacher does not preset a score, the large learning model outputs "AI scoring percentage: 80%" for a given question. After grading, the teacher sets the actual full score for that question to 10 points in the system. The system automatically calculates the actual score using the formula: Actual Score = 10 points * (80 / 100) = 8 points, and updates the final report.

[0046] like Figure 2As shown, an intelligent homework grading system based on a multimodal large model implements the above method through a modular architecture. It includes:

[0047] The image acquisition and preprocessing module is integrated into the tablet app and is responsible for taking pictures and automatic correction. The adaptive routing decision module, deployed on a cloud server, contains a content feature analysis unit and a decision engine based on a multimodal large model. The dual-modal grading engine is deployed on a cloud server and includes a visual grading component and a text grading component. The visual grading component calls the cloud multimodal big model API through the prompt word project to complete the question area location and grading score. The text grading component integrates the OCR engine and the big language model API and has built-in confidence judgment logic. The result compositing and rendering module is deployed within a tablet app. The entire system adopts a distributed architecture: students or teachers complete the taking and uploading of assignments on their local terminals (tablets).

[0048] After receiving the image, the cloud server sequentially performs routing decisions and bimodal corrections, then returns a structured result containing scores and coordinates via a RESTful API. The local terminal's rendering module uses the returned data to synthesize and display the final image with correction marks in real time, achieving efficient cloud-edge collaborative processing.

[0049] like Figure 5 As shown, the local terminal is responsible for scanning and uploading the work images, as well as synthesizing and displaying the final feedback image. The cloud server deploys a multimodal AI intelligent grading service with a built-in adaptive traffic splitting mechanism. It receives the image data uploaded by the local terminal, completes the grading, and returns structured information containing the scoring results and grading coordinates. The data interaction process is as follows: the local terminal uploads image data to the cloud, the cloud completes the grading and returns structured information, and the local terminal synthesizes the final feedback image based on this information.

[0050] The above-disclosed embodiments are merely preferred embodiments of the present invention. However, the embodiments of the present invention are not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for automatic homework grading and feedback based on a multimodal large model, characterized in that, include: Obtain images of students' handwritten assignments to be graded and corresponding answer key data; The system preprocesses images of students' handwritten assignments to identify and correct the rotation direction of the images; it then performs content feature analysis on the corrected images, including at least the proportion of graphic regions and the complexity of formulas, and generates routing decisions based on the analysis results. Based on the routing decision, the system adaptively selects to execute either a visual correction process or a text correction process. Execute the selected correction process: If the visual correction process is selected, the image region is input into the multimodal large model for visual reasoning to obtain the first structured output containing the score and the coordinates of the correction mark in the image. If the text correction process is selected, the text is first obtained by OCR recognition, and then the text is input into the processing model for semantic reasoning to obtain a second structured output containing a score and confidence level. If the confidence level of the text correction process output is lower than a preset threshold, the process will automatically revert and trigger the visual correction process to review the same content. Parse the first or second structured output to extract the scoring results and the location coordinates of the grading marks; Traceable rendering: Based on the location coordinate information, the corresponding visual correction mark is superimposed on the student's handwritten homework image to generate a traceable correction image, and the traceable correction image and the scoring report are output.

2. The method for automatic job grading and feedback based on a multimodal large model according to claim 1, characterized in that, The routing decision rules include: When the question area is detected to contain geometric figures, function graphs, or charts, it is determined to be a high-complexity visual task, and the routing decision is to execute the visual grading process. When the pixel density of the formula region in the corrected image exceeds the preset complexity threshold, it is determined to be a high-complexity visual task, and the routing decision is to execute the visual correction process. When the question area is detected to consist only of plain text or simple formulas, and no graphic area is detected, it is determined to be a text semantic task, and the routing decision is to execute the text grading process. For ambiguous situations between the two mentioned above, the image is pre-identified using OCR and a confidence level is obtained. If the confidence level is lower than a set threshold, the routing decision is to execute a visual correction process.

3. The method for automatic job grading and feedback based on a multimodal large model according to claim 1, characterized in that, The visual correction process specifically includes: Construct a prompt word for locating the question region. This prompt word contains the following instruction: "Please identify the answer region for each question in the assignment image and return the boundary coordinates of each region." Construct prompts for question location to guide a multimodal visual model to identify the answer area for each question and obtain its coordinate frame; Based on the coordinate frame, the student assignment image is cropped into individual question slice images; Construct a grading prompt word containing the following instructions: "Please analyze the problem-solving steps in the input image, determine whether they are correct or incorrect, return the relative coordinates of the grading mark for each step in the slice image of the problem, and identify the location coordinates of the final answer area for scoring and recording". Input the sequence of sliced ​​images of the question and the correction prompts into a multimodal large model; The visual encoder of the multimodal large model is used to perceive the handwritten trajectory and geometric structure, and output a JSON-formatted report containing the score, step correction coordinates and the coordinates of the final answer box.

4. The method of claim 3, wherein the method further comprises: When parsing a JSON report containing normalized coordinates, the local coordinates are restored to the full-image pixel coordinates of the student's handwritten homework image using the following formulas: Global X = Slice X + Normalized x × Slice width, Global Y = Slice Y + Normalized y × Slice height, thus restoring the local coordinates to the full-image pixel coordinates.

5. The method of claim 1, wherein the method further comprises: The text correction process specifically includes: The OCR engine is invoked to perform text detection and recognition on the entire assignment page to obtain full-text text information; Using a text layout analysis algorithm and combining question number features, the full text information is segmented into independent "question-answer" text pairs; Retrieve the standard answer text corresponding to the current question from the vector database; Construct a second multimodal cue word that includes semantic scoring and logical reasoning instructions; The "question-answer-response" triplet text is input into the second processing model for semantic reasoning to obtain the second structured output.

6. The method for automatic job grading and feedback based on a multimodal large model according to claim 1, characterized in that, The trace rendering also includes the following steps: Preload a library of correction icon materials in various handwritten styles; The scaling ratio of the correction icon is dynamically calculated based on the size and resolution of the student's handwritten homework image; By using the Alpha channel blending technique of the graphics processing library, the scaled correction icon layer is overlaid onto the corresponding coordinate position of the original homework image layer with a specified transparency, generating a visual composite image that resembles a teacher's handwritten corrections.

7. The method of claim 1, wherein the method further comprises: It also includes a scoring mechanism, specifically supporting the following two optional scoring modes: Preset score mode: Before the grading begins, the user pre-sets the score for each question and embeds the score information into the first or second multimodal prompt words, so that the large model directly outputs the actual score for each question based on the preset score. Percentage conversion mode: The multimodal large model or processing model outputs an AI score percentage for each question out of a full score of 100. After grading, the user sets the actual full score for each question, and the system automatically calculates the final score according to the formula "actual score = question score × (AI score percentage / 100)".

8. A multi-modal large model-based homework intelligent grading system for implementing the homework automatic grading and trace feedback method according to any one of claims 1 to 7, characterized in that, include: The image acquisition and preprocessing module is used to acquire images of students' handwritten homework to be graded and corresponding reference answer data, and to perform orientation detection and correction on the images; The adaptive routing decision module is used to perform content feature analysis on the corrected job images and generate routing decisions based on the analysis results to adaptively select the grading process. It includes an image detection unit, a formula analysis unit, and a decision engine. A dual-modal correction engine is used to execute the selected visual correction process or text correction process. It includes a visual correction component and a text correction component that work in parallel. The text correction component integrates a confidence judgment and rollback trigger unit for triggering the visual correction component to perform a review when the confidence level is insufficient. The results synthesis and rendering module is used to parse the output of the grading engine to extract scores and coordinates, and to overlay grading marks on the job image based on the coordinates, generating and outputting a grading image with traces and a grading report.

9. A smart homework correction system based on a multimodal large model according to claim 8, characterized in that, The visual correction component integrates a multimodal large model for receiving images and instructions and outputting structured data containing coordinates; the text correction component integrates an OCR engine and a text processing model. 10.The job intelligent grading system based on a multi-modal large model according to claim 8, wherein, The system adopts a distributed architecture that coordinates local terminals and cloud servers. The local terminal is responsible for scanning and collecting the images of the students' handwritten assignments, uploading them, and finally receiving, synthesizing, and displaying the images with the grading record. The cloud server is equipped with the adaptive routing decision module and the dual-modal grading engine, which are used to receive image data and complete intelligent grading, and return structured information containing scores and coordinates to the local terminal.

Images