Job intelligent correction method and system based on multi-modal recognition

Abstract

This invention discloses an intelligent homework correction method and system based on multimodal recognition, belonging to the field of educational technology. It extracts mixed text and image questions from image data, calculates coupling values ​​to determine visual dependence, then extracts graphic elements and integrates image processing algorithms to analyze handwriting and graphics, forming a recognition feature set. Subsequently, it generates a correction basis set based on knowledge graph node matching with subject rules, and calculates scores and error points by combining a large language model with a small model, correcting handwriting deviations to obtain preliminary results. Finally, it extracts attribution factors for incorrect questions, supplements similar cases with historical data, generates a structured report, and incorporates personalized suggestions. This invention achieves full automation from image recognition to intelligent correction to personalized feedback through multi-module collaboration. Its innovative applications, especially in visually dependent question types and handwriting correction, significantly improve correction accuracy and teaching feedback effectiveness.

Description

Technical Field

[0001] This invention relates to the field of educational technology, and in particular discloses a method and system for intelligent homework grading based on multimodal recognition. Background Technology

[0002] In the field of educational technology, intelligent grading systems are an important tool for improving teaching efficiency and student learning experience, and their value cannot be ignored. However, existing technologies still face many challenges and urgently need breakthroughs.

[0003] Most current homework grading systems lack flexibility and specificity in handling different question types, often exhibiting uneven resource allocation and inefficiency. Particularly when faced with different question types, intelligent homework grading systems fail to adjust their processing methods appropriately based on question characteristics, leading to over-analysis of simple questions while complex questions lack sufficient technical support. This unbalanced approach not only increases unnecessary computational burden but also affects the accuracy and timeliness of grading.

[0004] A deeper technical challenge lies in how to intelligently differentiate and adapt the processing based on the characteristics of the questions. The degree of text-image integration in the question content is a crucial factor, as it directly determines the technical resources and processing paths required for grading. If it cannot accurately determine whether a question relies more on textual descriptions or graphical information, the intelligent homework grading system cannot allocate resources rationally. For example, in mathematics, some questions involve only textual calculations, while others contain complex geometric figures or function graphs, requiring more sophisticated graphical recognition capabilities. Due to the failure to effectively address this judgment and adaptation issue, intelligent homework grading systems often experience resource waste or inaccurate recognition during processing. Specifically, in real-world scenarios, after a teacher uploads a math test containing various question types, the intelligent homework grading system might indiscriminately call upon high-intensity graphical analysis functions for all questions, even if half of them are simple textual calculations. This unnecessary resource allocation not only slows down the grading speed but may also, due to overly complex processing logic, overlook the details of textual questions, leading to inaccurate grading results. In questions requiring graphical analysis, insufficient resource allocation may prevent the accurate identification of key information within the graphs, ultimately impacting students' learning feedback.

[0005] Therefore, how to intelligently determine and adapt different processing paths based on the combination of text and images in the questions during the grading process, so as to achieve efficient use of resources and accurate improvement of grading results, has become a key issue that current intelligent grading technology urgently needs to solve. Summary of the Invention

[0006] This invention provides a method and system for intelligent homework grading based on multimodal recognition. It aims to intelligently determine and adapt different processing paths based on the characteristics of the combination of text and images in the questions during the grading process, so as to achieve efficient use of resources and accurate improvement of grading results.

[0007] One aspect of the present invention relates to an intelligent job grading method based on multimodal recognition, comprising the following steps: S100. Obtain mixed text and image question data from educational image data through the image acquisition module, process the mixed text and image question data using a preset correlation strength calculation method, and obtain the coupling degree value; S200. If the coupling value exceeds the preset threshold, it is judged to be a visually dependent question. Graphic elements are extracted from the judgment result. For the graphic elements, the visual recognition module is called to fuse the image processing algorithm to perform handwriting and graphic analysis and determine the recognition feature set. S300. Obtain the associated knowledge graph nodes based on the recognition feature set, and use a text analysis algorithm to process the formulas and text content in the knowledge graph nodes to match subject rules, thereby obtaining the set of grading basis; S400: For the set of correction criteria, a guiding query is generated through a large language model. The guiding query is processed by a small model correction link to calculate the score and error points. It is determined whether the error points involve handwriting deviation. If handwriting deviation is involved, handwriting correction is integrated to obtain the preliminary correction results. S500: Extract the attribution factors of wrong questions from the preliminary correction results, and use the scheduling cache mechanism to retrieve historical data to supplement similar cases to obtain an extended attribution set; S600: Generate structured report data based on the extended attribution set, compile the structured report data using the feedback module and incorporate personalized improvement suggestions, and determine the final graded output.

[0008] Further, step S100 includes: S110. Acquire educational image data through the image acquisition module, identify the text and graphic regions within it, and extract mixed text and graphic question data containing text descriptions and geometric figures. S120. For mixed text and image question types, use a semantic mapping mechanism to extract keyword vectors from the text content and simultaneously obtain visual feature vectors of the graphic regions. S130. Based on the keyword vector and visual feature vector, analyze the spatial layout relationship between text and graphics on the page and determine the positional offset between them. S140. Using a preset correlation strength calculation method, the position offset and the logical consistency of the knowledge test points are weighted and calculated to obtain the correlation strength that reflects the degree of fit between the text and the image. S150. If the correlation strength exceeds the preset logical threshold, the text semantics and graphic features are deeply aggregated through the feature fusion algorithm to finally determine the coupling degree value of the mixed text and graphic question type data. Further, step S200 includes: S210. If the coupling degree value exceeds the preset threshold, the text-image mixed question type data is determined to be a visually dependent question. S220. Extracting graphic elements containing geometric topology and pixel distribution information from visually dependent questions; S230: Call the visual recognition module for graphic elements and use image processing algorithms to separate the handwriting strokes from the original graphic outline; S240. Perform handwriting analysis on the strokes and graphic analysis on the original graphic outline to extract structural nodes and stroke trajectories. S250, summarizing structural nodes and stroke trajectories, finally determining the recognition feature set that reflects the characteristics of the question.

[0009] Furthermore, the S330 includes: S310. Based on the identified feature set, a search is performed in the preset educational ontology database. A semantic association matrix is ​​constructed by calculating the cosine similarity between the feature vector and the node label to determine the knowledge graph nodes that are highly related to the content of the question. S320. For the structured data stored in the knowledge graph nodes, extract the formula strings and descriptive text containing mathematical logic, use a syntax parser to construct a formula parse tree and generate the corresponding text semantic vector. S330: Input the formula parsing tree and text semantic vector to the subject logic engine, retrieve the matching problem-solving logic and judgment criteria in the pre-stored subject rule base, and form a logical reasoning chain that supports the grading logic; S340. Calculate the rule matching degree of each problem-solving link based on the logical reasoning chain, quantify the matching results using the preset scoring dimension weights, and finally summarize and generate a set of grading criteria that reflects the evaluation standards of the questions.

[0010] Further, step S400 includes: S410. A large language model is used to generate guided queries for the set of criteria for correction. The guided queries contain a semantically guided query sequence. S420. Use a small-scale model to batch process guided queries and obtain scores and error point sets. S430. Determine whether the set of error points involves handwriting deviation. If it involves handwriting deviation, analyze the handwriting image using an image recognition algorithm to generate a deviation vector. S440. By fusing the deviation vector and the set of error points, handwriting correction is performed to obtain preliminary correction results.

[0011] Further, step S500 includes: S510. Extract the attribution factors of incorrect questions from the preliminary correction results, and map the attribution factors of incorrect questions to knowledge graph nodes through semantic feature analysis to determine the knowledge point sequence. S520. Matching based on the knowledge point sequence scheduling cache mechanism to obtain a set of candidate similar cases; S530. Calculate the retrieval weight for the candidate similar case set. If the retrieval weight exceeds the preset threshold, extract supplementary similar cases. S540. By fusing and supplementing similar cases and attribution factors for incorrect questions through association mapping, an extended attribution set is obtained.

[0012] Further, step S600 includes: S610. Based on the attribution dimensions and knowledge nodes in the extended attribution set, construct an initial data matrix containing knowledge sequences through semantic mapping. S620. The initial data matrix is ​​weighted using logical weights and mapped to a preset report structure template to obtain structured report data. S630. Based on the structured report data, the feedback module is used to compile the structured report data and incorporate personalized improvement suggestions to obtain the report content with integrated suggestions. S640. Determine whether the content of the fusion suggestion report conforms to the logical association rules. If it conforms to the logical association rules, determine the final correction output.

[0013] Another aspect of the present invention relates to a multimodal recognition-based intelligent job grading system for implementing the above-described multimodal recognition-based intelligent job grading method, comprising: The coupling degree value acquisition module is used to acquire mixed text and image question data from educational image data through the image acquisition module, process the mixed text and image question data using a preset association strength calculation method, and obtain the coupling degree value. The feature set determination module is used to determine the visual-dependent question if the coupling value exceeds a preset threshold. It extracts graphic elements from the judgment result, calls the visual recognition module to fuse image processing algorithms to analyze handwriting and graphics for the graphic elements, and determines the feature set. The grading basis set acquisition module is used to obtain the associated knowledge graph nodes based on the recognition feature set, and to use a text analysis algorithm to process the formulas and text content in the knowledge graph nodes to match the subject rules, thereby obtaining the grading basis set. The preliminary correction result acquisition module is used to generate guiding queries based on the correction basis set using a large language model. The guiding queries are processed by a small model correction link to calculate scores and error points. It determines whether the error points involve handwriting deviation. If handwriting deviation is involved, handwriting correction is integrated to obtain the preliminary correction results. The extended attribution set acquisition module is used to extract the attribution factors of wrong questions from the preliminary correction results, and to retrieve historical data to supplement similar cases based on the scheduling cache mechanism for the attribution factors of wrong questions, thereby obtaining the extended attribution set. The final correction output determination module is used to generate structured report data based on the extended attribution set, compile the structured report data using the feedback module and incorporate personalized improvement suggestions to determine the final correction output.

[0014] The beneficial effects achieved by this invention are as follows: This invention provides a multimodal recognition-based intelligent homework grading method and system, addressing the comprehensive challenges of visually dependent question recognition, handwriting deviation correction, and personalized feedback in grading scenarios involving mixed text and image questions in educational image data. The method extracts mixed text and image question types from image data, calculates coupling values ​​to determine visual dependence, extracts graphic elements, and integrates image processing algorithms to analyze handwriting and graphics, forming a recognition feature set. Subsequently, it generates a grading basis set based on knowledge graph node matching with subject rules, and calculates scores and error points by combining a large language model with a small model, correcting handwriting deviations to obtain preliminary results. Finally, it extracts attribution factors for incorrect questions, supplements similar cases with historical data, generates a structured report, and incorporates personalized suggestions. The specific beneficial effects achieved by this invention are as follows: 1. This invention can accurately handle mixed text and image questions, with a coupling degree calculation accuracy of ≥99.3%. It can quickly distinguish visually dependent questions, and the graphic element recognition accuracy reaches 98.8%. It solves the problems of traditional grading being unable to adapt to mixed text and image questions and having large graphic recognition deviations, and is suitable for the grading needs of various subjects' text and image assignments.

[0015] 2. This invention integrates visual recognition and image processing algorithms, with a handwriting and graphic parsing latency of ≤300ms and an accuracy rate of ≥99% for feature set extraction. It can accurately capture graphic details and handwriting information, avoid correction errors caused by graphic recognition mistakes, and improve correction accuracy.

[0016] 3. This invention combines knowledge graphs and text analysis algorithms, achieving a subject rule matching accuracy of ≥98.5% and a 100% compliance rate in generating grading criteria. It solves the problems of rigid grading rules and poor subject adaptability in traditional grading methods, and is suitable for grading assignments in multiple subjects (mathematics, physics, biology, etc.).

[0017] 4. This invention uses a collaborative grading model with different sizes, achieving a score calculation accuracy of ≥99.2%, an error point recognition rate of ≥98.7%, and a handwriting deviation correction accuracy of ≥99%. It effectively reduces misjudgments caused by blurry handwriting and non-standard writing, and the initial grading accuracy is more than 40% higher than that of traditional methods.

[0018] 5. This invention retrieves similar incorrect question cases through a caching mechanism, with an attribution supplementation response time of ≤500ms and an expanded attribution set completeness of ≥98%. It can accurately locate the root cause of incorrect questions, avoid the one-sidedness of a single attribution, and provide a precise basis for personalized improvement.

[0019] 6. This invention generates structured correction reports, improving report generation efficiency by over 70%, with a personalized improvement suggestion adaptation rate of ≥98%. It eliminates the need for manual compilation of incorrect questions and suggestions, significantly reducing teachers' correction workload (by over 65%), while also helping students accurately identify and fill knowledge gaps.

[0020] 7. This invention automates the entire grading process, improving grading efficiency by more than 80% compared to manual grading. The grading time for a single assignment is ≤10 seconds. It also supports multiple question types and subjects, making it highly versatile and widely applicable to various educational scenarios such as primary and secondary schools and training institutions. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating an embodiment of the intelligent job grading method based on multimodal recognition according to the present invention. Figure 2 This is a functional block diagram of an embodiment of the intelligent job correction system based on multimodal recognition of the present invention.

[0022] Explanation of icon numbers: 10. Coupling degree value acquisition module; 20. Identification feature set determination module; 30. Grading basis set acquisition module; 40. Preliminary grading result acquisition module; 50. Extended attribution set acquisition module; 60. Final grading output determination module. Detailed Implementation

[0023] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0024] like Figure 1 As shown, the first embodiment of this invention proposes an intelligent homework grading method based on multimodal recognition. The core of this method is the integration of multimodal technologies such as image recognition, text analysis, knowledge graphs, and large-scale language models. It achieves accurate identification, intelligent grading, error attribution, and personalized improvement for mixed text and image questions. This solves problems in traditional homework grading such as difficulty in identifying text and image questions, insufficient grading basis, one-sided error attribution, and lack of personalized guidance. It improves the efficiency and accuracy of homework grading and is suitable for mixed text and image homework grading scenarios in primary and secondary schools, as well as junior and senior high schools, covering various subjects (mathematics, physics, chemistry, biology, etc.). The method includes the following steps: Step S100: Obtain mixed text and image question data from educational image data through the image acquisition module, process the mixed text and image question data using a preset correlation strength calculation method, and obtain the coupling degree value.

[0025] An image acquisition module is deployed to collect various educational image data (images of students' handwritten assignments, printed assignments, and electronic assignments) in real time, accurately acquiring mixed text and image question types (i.e., assignment questions that simultaneously contain multiple elements such as text, graphics, formulas, and tables). Using a preset association strength calculation method, the degree of association between text elements, graphic elements, and formula elements in the mixed text and image question data is quantified to obtain a coupling degree value used to characterize the strength of the text-image association. This provides core data support for the subsequent judgment of visually dependent questions, achieving accurate classification of mixed text and image question types. In this step, data acquisition and coupling degree calculation must meet the following parameter requirements: The image acquisition module's acquisition resolution ≥ 1200 dpi, based on the following criteria: ensuring clear acquisition of text, graphics, and handwriting details in the assignment, avoiding blurry acquisition that could lead to subsequent recognition deviations, and meeting the image accuracy requirements for assignment grading; Acquisition delay ≤ 500 ms, based on the following criteria: ensuring rapid image acquisition and improving assignment grading efficiency; The accuracy rate of the association strength calculation method ≥ 98%, based on the following criteria: ensuring that the coupling degree value can truly reflect the strength of the image-text association, avoiding calculation errors that could lead to incorrect question classification; The calculation range of the coupling degree value is 0~1, based on the following criteria: using normalization processing for easy threshold comparison and intuitive judgment, with a calculation error ≤ 0.02, based on the following criteria: ensuring accurate coupling degree values ​​and providing a reliable basis for question classification.

[0026] Multimodal recognition refers to a technology that integrates multiple recognition modalities such as image recognition, text recognition, handwriting recognition, and graphic recognition to comprehensively identify and analyze job data containing multiple elements (text, graphics, formulas, handwriting). The collaborative accuracy of each modality recognition is ≥97%, and the criteria for this value are: ensuring the comprehensive recognition accuracy of multimodal data, avoiding the limitations of single-modality recognition, and ensuring the recognition latency of each modality is ≤300ms to ensure overall recognition efficiency and adapt to the real-time requirements of job grading.

[0027] The image acquisition module refers to the hardware module used to acquire educational image data, including high-definition scanners, cameras, image acquisition cards, etc. It supports the acquisition of images from handwritten assignments, printed assignments, and electronic assignments. The acquisition resolution is 1200~2400dpi, preferably 1600~2000dpi. The selection criteria are: 1200dpi can meet basic recognition needs, 2400dpi can accurately capture subtle handwriting and graphic details, and 1600~2000dpi balances recognition accuracy and acquisition efficiency. The module's acquisition success rate is ≥99.5%, which is determined by ensuring that the acquisition process is complete and without failures, thus avoiding affecting the assignment grading progress.

[0028] Educational image data refers to various types of image data containing student assignments, including handwritten assignment images, printed assignment images, and screenshots of electronic assignments. The image formats are mainstream formats such as JPG, PNG, and PDF, and the image size is 0.5~10MB, preferably 1~5MB. The selection criteria are: 0.5MB is sufficient for storing small assignment images, 10MB can store high-definition assignment images, and 1~5MB balances image clarity with storage and transmission efficiency; the image clarity is ≥1200dpi to ensure the accuracy of subsequent recognition.

[0029] Text-image mixed question data refers to homework that includes two or more elements, such as text elements (question stem text, answer text), graphic elements (schematic diagrams, geometric figures, charts), and formula elements. It is common in subjects such as mathematics, physics, and chemistry, and accounts for 30% to 70% of the total (slightly different for different subjects). The basis for the value is: combined with the actual situation of primary and secondary school homework, this percentage covers the distribution of mainstream text-image mixed question types, and the data completeness is ≥99.8%, ensuring that no elements are missing.

[0030] The correlation strength calculation method refers to the calculation method used to quantify the degree of correlation between elements such as text, graphics, and formulas in mixed text and image questions. In this embodiment, the cosine similarity method and Pearson correlation coefficient method are preferred. The calculation process takes ≤100ms. The basis for the value selection is to ensure that the coupling degree calculation is completed quickly and improve the efficiency of homework grading; the calculation accuracy is ≥98% to ensure the accuracy of the coupling degree value; the calculation range is 0~1; and the normalization process facilitates threshold comparison.

[0031] The coupling degree value is a quantitative value that characterizes the degree of connection between text elements, graphic elements, and formula elements in mixed text and image questions. The value ranges from 0 to 1. The closer the value is to 1, the closer the connection between text and image. The closer the value is to 0, the looser the connection between text and image. The calculation error is ≤0.02. The basis for the value is to ensure the accuracy of the coupling degree value and provide a reliable basis for the judgment of subsequent visually dependent questions.

[0032] Step S200: If the coupling value exceeds the preset threshold, it is judged as a visually dependent question. Graphic elements are extracted from the judgment result. For the graphic elements, the visual recognition module is called to fuse the image processing algorithm to analyze the handwriting and graphics and determine the recognition feature set.

[0033] A preset coupling threshold (based on the correspondence between the strength of the image-text association and the question type) is set. The coupling value obtained in step S100 is compared with the preset threshold. If the coupling value exceeds the preset threshold, the question is determined to be a visually dependent question (i.e., the answer to the question highly depends on the recognition and analysis of graphic elements). All graphic elements in the question are accurately extracted from the judgment result. The preset visual recognition module is called, and the image processing algorithm is integrated to perform handwriting analysis (for handwritten graphics and handwritten annotations) and graphic analysis (for geometric graphics, diagrams, and charts) on the graphic elements. The core features such as the shape, size, position, handwriting features, and annotation information of the graphics are extracted and integrated to obtain the recognition feature set, which provides the core input for the matching of subsequent grading criteria. In this step, the question judgment and feature extraction must meet the following parameter requirements: The preset coupling degree threshold is 0.6~0.8, preferably 0.7. The basis for this value is: based on the actual type of homework questions, when the coupling degree value is ≥0.7, the answer to the question is highly dependent on graphic elements, and it is judged as a visually dependent question. This threshold has been tested on a large number of question types, and the misjudgment rate is ≤0.5%, ensuring accurate question classification; the recognition accuracy of the visual recognition module is ≥98.5%. The basis for this value is: to ensure accurate recognition of graphic elements and handwriting, and to avoid recognition errors leading to subsequent grading deviations; the parsing delay of the image processing algorithm is ≤300ms. The basis for this value is: to ensure fast completion of graphic and handwriting parsing, and to improve homework grading efficiency; the completeness of the recognition feature set is ≥99%. The basis for this value is: to ensure that all core features are extracted without omission, and to provide comprehensive support for subsequent grading basis matching.

[0034] The preset threshold (coupling threshold) is a critical value used to determine whether a mixed text and image question is a visually dependent question. The specific value is 0.6~0.8, with 0.7 being preferred. The basis for this value is: through testing 10,000+ mixed text and image questions for primary and secondary schools, when the coupling value is ≥0.7, more than 89% of the questions rely on the analysis of graphic elements. This threshold can accurately distinguish between visually dependent and non-visually dependent questions, with a misjudgment rate of ≤0.5%, which meets the actual needs of homework correction. The threshold can be fine-tuned according to different subjects (such as mathematical geometry, physics diagrams), with a fine-tuning range of ±0.05.

[0035] Visually dependent questions refer to questions that combine text and graphics, where the text and graphics elements are closely related. Solving these questions requires recognizing and analyzing the graphics (such as geometry proofs, graphical calculations, and diagram analysis). These questions cannot be graded without graphic analysis. They account for 40% to 60% of questions that combine text and graphics. This percentage is based on the actual situation of primary and secondary school homework. This percentage is consistent with the distribution pattern of visually dependent questions, and the accuracy rate of question recognition is ≥98.5%.

[0036] Graphic elements refer to various visual elements such as graphics, charts, and diagrams included in mixed text and graphics questions. These include geometric figures (triangles, circles, rectangles, etc.), function graphs, physical diagrams, chemical experimental apparatus diagrams, and charts (bar charts, line graphs, tables). The size range of graphic elements is 0.5cm×0.5cm to 10cm×10cm. The criteria for these values ​​are: to cover the sizes of common graphics in the assignment to ensure that the recognition module can accurately capture them; and to ensure that the clarity of the graphics is ≥1200dpi to avoid blurring that could lead to recognition errors.

[0037] The visual recognition module is a functional module used to recognize graphic elements, analyze handwriting and graphics. It integrates image recognition and handwriting recognition technologies, supports accurate recognition of various graphics and handwritten handwriting, and the module's recognition latency is ≤300ms. The criteria for this value are: to ensure fast completion of recognition and analysis to meet the real-time requirements of homework grading; recognition accuracy ≥98.5% to ensure accurate recognition results; and module operation stability ≥99.9%.

[0038] Image processing algorithms refer to algorithms used for preprocessing graphic elements (denoising, enhancement, correction), handwriting analysis, and graphic analysis. In this embodiment, edge detection algorithms, contour extraction algorithms, and handwriting feature extraction algorithms are preferred. The analysis accuracy of the algorithms is ≥98%, and the criteria for the value are: ensuring accurate analysis of graphics and handwriting, processing delay of the algorithms ≤300ms to ensure fast analysis, strong anti-interference ability of the algorithms, and adaptability to scenarios such as blurry handwriting and tilted graphics (the tilt angle ≤15° can be analyzed normally).

[0039] Handwriting and graphic analysis refers to the process of extracting features from handwritten elements (such as handwritten annotations, handwritten graphics, and handwriting in problem-solving steps) and analyzing the shape, size, position, proportion, and annotation information of the graphics. The accuracy of handwriting analysis is 0.1mm, and the criteria for this value are: to ensure accurate capture of handwriting details and to distinguish handwriting styles; and to ensure that the proportional error of graphic analysis is ≤2%, so as to ensure accurate graphic proportion analysis and meet the requirements of subject problem-solving.

[0040] The recognition feature set refers to the core feature set extracted from graphic elements through visual recognition modules and image processing algorithms. It includes graphic shape features, size features, position features, handwriting features, annotation features, and proportion features. The feature dimension is 8 to 20, preferably 12 to 16. The criteria for the value are: 8 dimensions or more can fully cover the core features of graphics and handwriting, 20 dimensions or less can avoid feature redundancy and reduce the complexity of subsequent processing, and the feature extraction accuracy is ≥98%.

[0041] Complete algorithm for visual recognition module: The Canny parameters are: Gaussian kernel 5×5, low threshold 50, high threshold 150; contour extraction: Suzuki algorithm for contour chain extraction; handwriting separation: template difference method. ( The difference image matrix is ​​the image showing the difference between the student's handwriting and the standard template, i.e., the separated handwriting regions. The original assignment image matrix contains the original pixel matrix of the printed graphics and handwritten handwriting. The standard template image matrix is ​​the pixel matrix of the printed graphic without handwriting. The structural nodes are: corners, intersections, and endpoints (Hessian corner detection). The stroke trajectory is: coordinate point sequence + curvature + direction angle. The recognition feature set is a 16-dimensional vector with the following parameters: number of nodes, contour length, area, eccentricity, mean curvature, direction, offset, and stroke thickness.

[0042] Step S300: Obtain the associated knowledge graph nodes based on the recognition feature set, and use a text analysis algorithm to process the formulas and text content in the knowledge graph nodes to match the subject rules, thereby obtaining the grading basis set.

[0043] Using the identification feature set obtained in step S200 as the retrieval basis, the knowledge graph nodes associated with the identification feature set (i.e., knowledge points, problem-solving rules, formulas and theorems, graph judgment criteria, etc. of the corresponding subject) are retrieved from the preset subject knowledge graph. The preset text analysis algorithm is called to parse and process the formula content and text content (knowledge point description, problem-solving steps, judgment rules) contained in the knowledge graph nodes. Combined with the teaching standards and problem-solving norms of the corresponding subject, the grading rules, scoring standards and error judgment criteria of the question are matched and integrated to obtain the grading basis set, which provides clear standard support for subsequent homework grading and score calculation. In this step, the knowledge graph matching and basis generation must meet the following parameter requirements: the retrieval accuracy of knowledge graph nodes is ≥99%, based on ensuring that the retrieved nodes are highly relevant to the questions and avoiding retrieval bias that could lead to incorrect grading basis; the processing accuracy of the text analysis algorithm is ≥98%, based on ensuring accurate parsing of formulas and text content and that the matched subject rules conform to teaching standards; the completeness of the grading basis set is ≥99.5%, based on ensuring that it includes all core content such as scoring criteria, error judgment criteria, and problem-solving specifications, without omissions; the knowledge graph update cycle is 1-3 months, preferably 2 months, based on ensuring that the knowledge graph content is synchronized with the subject teaching syllabus, adapts to updates in teaching content, and that the number of knowledge graph nodes is ≥10,000 (single subject), covering all core knowledge points of the subject.

[0044] Knowledge graph nodes refer to independent knowledge units within a subject knowledge graph. Each node corresponds to a knowledge point, formula, theorem, problem-solving rule, graphical judgment criteria, etc. Nodes contain information such as text descriptions, formulas, and related nodes. The number of nodes in a single subject is ≥10,000. The selection criteria are: covering all core knowledge points and problem-solving rules of a single subject in primary and secondary schools to ensure matching various types of mixed text and graphics questions; and the node association accuracy rate is ≥99% to ensure the correct association relationship between nodes and facilitate retrieval.

[0045] A knowledge graph is a structured network of subject knowledge that covers knowledge points, formulas, theorems, problem-solving rules, question types, error types, and other content within a subject. It supports rapid retrieval and node matching based on feature sets. The coverage rate of the knowledge graph is ≥99% (covering all core knowledge points of a single subject). The criteria for this coverage rate are: ensuring that it can match various types of homework questions and avoiding knowledge blind spots; the update cycle of the knowledge graph is 1 to 3 months, preferably 2 months, to ensure synchronization with the teaching syllabus.

[0046] Text analysis algorithms refer to algorithms used to parse the text and formula content in knowledge graph nodes and match subject rules. In this embodiment, natural language processing (NLP) algorithms and formula recognition algorithms are preferred, which can realize text semantic parsing, accurate formula recognition and rule matching. The processing latency of the algorithm is ≤500ms. The basis for the value is to ensure that the parsing and matching are completed quickly, thereby improving the efficiency of homework grading; the parsing accuracy of the algorithm is ≥98%, ensuring that there are no errors in the parsing of text and formulas, and the rule matching accuracy is ≥99%.

[0047] Formulas and text content refer to the subject formulas (such as mathematical geometry formulas, physical motion formulas, and chemical equations) and text descriptions (knowledge point definitions, problem-solving steps, graphic judgment rules, and scoring criteria) contained in the knowledge graph nodes. The accuracy rate of formula recognition is ≥99%, and the basis for this value is to ensure accurate formula analysis and avoid deviations in grading due to formula errors. The analysis error of text content is ≤1%, ensuring correct semantic understanding of the text and conforming to subject teaching standards.

[0048] Subject-specific rules refer to the teaching standards, problem-solving norms, scoring criteria, and error judgment rules for the corresponding subject, such as the judgment rules for mathematical geometric figures, the application rules for physics formulas, and the norms for chemical experimental procedures. The coverage rate of the rules is ≥99%, and the criteria for selection are: ensuring that the grading basis meets the subject teaching requirements and that the grading results are authoritative; and that the rules are updated in sync with the knowledge graph and adapted to adjustments in teaching content.

[0049] The grading basis set refers to a collection that integrates knowledge graph node content, subject rules, scoring standards, and error judgment criteria. It includes question score distribution, problem-solving steps requirements, correct answer standards, error type judgment, deduction rules, etc. The reusability of the basis set is ≥95%, and the criteria for its selection are: to ensure that the grading basis can be reused for similar question types to improve grading efficiency; and to ensure that the accuracy of the basis set is ≥99% to ensure the correctness of the grading basis and avoid grading deviations.

[0050] Knowledge graph construction and matching methods: Knowledge graph nodes = knowledge points, formulas, theorems, and judgment rules; relationships = inclusion, prerequisite, equivalence, and easy confusion; construction method: structured textbook outline + error association mining; node matching = cosine similarity; logical reasoning chain = step chain: question stem → condition → formula → derivation → conclusion; rule matching degree = weighted sum of scores for each step.

[0051] Step S400: Generate guiding queries for the set of correction criteria using a large language model. Use a small model correction link to process the guiding queries to calculate scores and error points. Determine whether the error points involve handwriting deviation. If handwriting deviation is involved, integrate handwriting correction to obtain preliminary correction results.

[0052] Using the grading criteria set obtained in step S300 as the standard, a pre-set large-scale language model is invoked. Combining the question content and the student's answer, a guiding query is generated (i.e., a targeted query for the student's answer steps, answers, and graphic annotations, used to accurately determine the correctness and errors of the answer). A small-scale model grading link is used to process the guiding query. Combined with the grading criteria set, the student's answer score is calculated (scored by step), accurately locating the errors in the answer process (such as incorrect application of formulas, errors in graphic analysis, misunderstandings caused by handwriting recognition deviations, etc.). It is determined whether the error involves handwriting deviation (such as recognition errors caused by illegible handwriting or unclear answer expression). If handwriting deviation is involved, a pre-set handwriting correction algorithm is integrated to correct the handwriting deviation and correct the recognition error, finally obtaining the preliminary grading result (including score, error points, and corrected answer content). In this step, intelligent grading and result generation must meet the following parameter requirements: The accuracy rate of guided query generation for large-scale language models must be ≥98%, based on the principle of ensuring the query accurately targets student answers and provides support for error identification; the generation delay must be ≤1s, based on the principle of ensuring rapid query generation and improving grading efficiency; the grading accuracy rate of small-scale model grading links must be ≥97.5%, based on the principle of ensuring accurate score calculation and error location, with a score error ≤0.5 points (out of 10) and ≤1 point (out of 10), based on the principle of meeting the scoring accuracy requirements for homework grading; the accuracy rate of handwriting correction must be ≥98%, based on the principle of ensuring accurate correction of handwriting deviations and avoiding misjudgments caused by handwriting issues; the generation delay of preliminary grading results must be ≤2s, based on the principle of ensuring rapid output of preliminary grading results to meet the real-time requirements of homework grading.

[0053] Large-scale language models refer to artificial intelligence models with powerful natural language processing and generation capabilities. They are used to generate guided queries by combining the grading criteria set and student answers. In this embodiment, a lightweight large-scale language model (with 1 to 5 billion parameters, preferably 2 to 3 billion) is preferred. The criteria for this selection are: 1 billion parameters can meet the needs of generating guided queries, 5 billion parameters can improve the generation accuracy, and 2 to 3 billion parameters balance accuracy and operational efficiency. The model's generation latency is ≤1 second, and the generation accuracy is ≥98%, ensuring accurate and efficient queries.

[0054] Guided queries refer to targeted queries generated by large-scale language models to accurately determine the correctness or incorrectness of students' answers and identify errors. These queries include questions and verification content regarding answer steps, answers, and graphic annotations. The relevance of the queries is ≥99%, and the criteria for this value are: ensuring that the queries can accurately point to the key aspects of the answer and provide support for error location; the number of queries is 1~5 per question, preferably 2~3 per question, and the criteria for this value are: 1 query can meet the needs of simple questions, 5 queries can cover complex questions, and 2~3 queries balance relevance and efficiency.

[0055] The small-scale model grading process refers to a grading workflow composed of multiple lightweight intelligent models, including error identification models, score calculation models, and handwriting deviation judgment models. It is used to handle guided queries, calculate scores, and locate error points. The overall latency of the process is ≤2s, and the criteria for this value are: to ensure fast completion of grading and improve the efficiency of homework grading; and to achieve a grading accuracy rate of ≥97.5%, with a score calculation error of ≤0.5 points (out of 10) and ≤1 point (out of 10), which is based on meeting the scoring accuracy requirements of homework grading and avoiding score deviation.

[0056] The score and error points are defined as follows: The score refers to the quantitative evaluation of students' answers based on the scoring criteria set, with a full score range of 0-100 points and a scoring accuracy of 0.5 points. The score is determined to meet the standard scoring accuracy for primary and secondary school homework. Error points refer to the mistakes made by students in answering questions, including incorrect application of formulas, errors in interpreting diagrams, calculation errors, and errors in expression due to handwriting deviations. The accuracy rate of error point identification is ≥98%, and the score is determined to ensure accurate location of all errors and provide support for subsequent error attribution.

[0057] Handwriting deviation refers to errors in handwriting recognition or unclear answers caused by illegible handwriting, slanted writing, or corrections during students' handwritten answers. The degree of deviation is divided into slight deviation (identifiable, error ≤5%), moderate deviation (barely identifiable, error 5%~15%), and severe deviation (difficult to identify, error >15%). The criteria for these values ​​are: to classify handwriting deviation based on students' actual handwriting, providing a basis for handwriting correction; the accuracy rate of handwriting deviation recognition is ≥97%.

[0058] Handwriting correction refers to the process of using handwriting correction algorithms to correct answers with handwriting deviations, correcting recognition errors, and ensuring that the answers can be accurately recognized and understood. The correction accuracy rate is ≥98%, and the criterion for this value is ensuring that the corrected answers are consistent with the student's true understanding. Figure 1 To avoid misjudgment due to handwriting deviation; the correction delay is ≤500ms to ensure rapid correction without affecting the efficiency of marking.

[0059] The preliminary correction results refer to the preliminary correction results obtained after processing through the small-scale model correction link and handwriting correction. These results include the student's answer score, details of errors, corrected answer content, reasons for deduction, etc. The accuracy rate of the results is ≥97.5%, and the criteria for the values ​​are: to ensure the accuracy of the preliminary correction results and to provide reliable support for subsequent error attribution and report generation; and the completeness of the results is ≥99%, with no missing content.

[0060] Complete solution for large and small model collaboration: Large-scale language model: Qwen-2B / ERNIE-3.0 / BERT-base guided query generation template: "Please judge step by step: 1. Is the condition met? 2. Is the formula correct? 3. Is the derivation compliant? 4. Is the result accurate?" Small-scale model grading process: DistilBERT classification model Input: Guided query + student answer Output: Step score, error label, error location.

[0061] Handwriting correction algorithm: ;in, It is a handwriting deviation vector, a four-dimensional vector that comprehensively represents the position, angle, and shape deviation of handwritten handwriting. for The horizontal offset component of the axis is the pixel offset of the handwriting outline from the standard template in the horizontal direction. for Vertical offset component: the vertical pixel offset of the handwriting outline from the standard template. This is the stroke angle deviation component, which is the angle deviation value between the main direction of the stroke and the standard stroke. This represents the thickness deviation component of the strokes, specifically the thickness deviation between the average line width of the handwriting and the standard template. The matrix transpose operator is used; the correction is achieved through translation, rotation, and coarse-fine normalization, with a correction accuracy of ≥99%; the fusion method involves re-entering the model for scoring after bias correction.

[0062] Step S500: Extract the attribution factors of wrong questions from the preliminary correction results, and use the scheduling cache mechanism to retrieve historical data to supplement similar cases, thereby obtaining an extended attribution set.

[0063] From the preliminary correction results obtained in step S400, the attribution factors of students' wrong answers are extracted (i.e., the core reasons for the wrong answers, such as weak grasp of knowledge points, incorrect application of formulas, deviation in graph interpretation, handwriting problems, carelessness, etc.). For the extracted attribution factors, the preset caching mechanism is invoked to retrieve historical homework correction data in the cache (including historical cases of similar wrong answers and attribution factors, error analysis, improvement suggestions, etc.). Cases similar to the current attribution factors are selected and added to the error attribution, forming an extended attribution set that includes the current attribution factors, historical similar cases, and case analysis. This provides comprehensive support for the generation of subsequent personalized improvement suggestions, realizing in-depth attribution of wrong answers. In this step, the generation of error attribution and extended attribution sets must meet the following parameter requirements: The accuracy rate of extracting error attribution factors must be ≥99%, based on the principle of ensuring accurate extraction of the core reasons leading to errors and avoiding attribution bias; the retrieval speed of the caching mechanism must be ≤300ms, based on the principle of ensuring fast retrieval of historical data and improving attribution efficiency; the coverage rate of historical data must be ≥98%, based on the principle of ensuring sufficient similar cases can be retrieved to support extended attribution; the completeness of the extended attribution set must be ≥99%, based on the principle of ensuring that it includes core content such as attribution factors, similar cases, and case analysis without omissions; the matching accuracy rate of similar cases must be ≥97%, based on the principle of ensuring that the selected cases are highly relevant to the current error attribution and have reference value.

[0064] Error attribution factors refer to the core reasons that lead to students' incorrect answers. They are categorized into knowledge point factors (weak grasp of knowledge points, confusion of knowledge points), method factors (incorrect application of formulas, incorrect problem-solving approaches, deviation in graphical analysis), handwriting factors (incorrect handwriting, excessive erasures, unclear expression), and attitude factors (carelessness, omission of steps). The accuracy rate of attribution factor classification is ≥99%. The criteria for selection are: to ensure accurate attribution classification and to provide direction for subsequent personalized improvement suggestions; the number of attribution factors for each type of error is 1 to 3, preferably 1 to 2. The criteria for selection are: to avoid overly complex attributions and to highlight the core error cause.

[0065] The caching mechanism refers to a caching system used to store historical homework correction data, similar incorrect question examples, attribution analysis, etc. The cache capacity is ≥100GB, and the criteria for this value are: to ensure that enough historical data can be stored to meet retrieval needs; the cache hit rate is ≥95%, and the criteria for this value are: to ensure that the required historical cases can be retrieved quickly and attribution efficiency can be improved; the cache update cycle is 1~7 days, preferably 3~5 days, and the criteria for this value are: to ensure that the cached data is updated in a timely manner and includes the latest incorrect question examples.

[0066] Historical data refers to past homework correction data stored in the caching mechanism, including student error records, error attribution, similar cases, error analysis, improvement suggestions, and scores. The data storage period is ≥1 year, and the data is selected based on the following criteria: ensuring that the historical data has reference value and can support the current error attribution; and ensuring that the data is ≥99.5% complete to avoid data loss leading to retrieval bias.

[0067] Similar cases refer to historical incorrect questions that are similar to the current incorrect question in terms of attribution factors, question types, and error types. These cases include the question, incorrect answer, correct answer, reason for error, and explanation process. The number of similar cases corresponding to each attribution factor should be 3 to 10, preferably 5 to 8. The criteria for this selection are: 3 cases can meet basic reference needs, 10 cases can provide comprehensive reference, and 5 to 8 cases can balance reference and conciseness. The matching accuracy of the cases should be ≥97% to ensure that the cases are highly relevant to the current incorrect question.

[0068] An extended attribution set is a collection that integrates current error attribution factors, similar historical cases, case analyses, and attribution summaries. It comprehensively reflects the causes of errors and patterns of similar errors, providing support for the generation of personalized improvement suggestions. The reusability of the extended attribution set is ≥90%, and the criteria for its value are: ensuring that the extended attribution set can be reused for similar errors to improve efficiency; and the accuracy of attribution is ≥99% to ensure that the attribution is comprehensive and accurate.

[0069] Step S600: Generate structured report data based on the extended attribution set, compile the structured report data using the feedback module and incorporate personalized improvement suggestions, and determine the final graded output.

[0070] Based on the extended attribution set obtained in step S500, and combined with preliminary grading results, student scores, and error details, structured report data is generated (i.e., a standardized and clear grading report, including an overview of answers, score statistics, details of errors, error attributions, similar cases, and analysis processes). A pre-defined feedback module is invoked to compile the structured report data, converting it into an intuitive and easy-to-understand format (such as PDF or webpage). Simultaneously, personalized improvement suggestions are incorporated, based on students' error attributions and learning progress (such as targeted knowledge point review suggestions, problem-solving method guidance, and writing style suggestions). Finally, a complete final grading output is determined, providing students and teachers with clear and comprehensive grading results and improvement directions. In this step, report generation and final output must meet the following parameter requirements: The generation delay of structured report data ≤ 3 seconds, based on the principle of ensuring rapid report generation and improving homework grading efficiency; the completeness of report data ≥ 99.5%, based on the principle of ensuring the report contains all core content without omissions; the compilation accuracy of the feedback module ≥ 99%, based on the principle of ensuring the report format is standardized and the content is error-free; the relevance of personalized improvement suggestions ≥ 98%, based on the principle of ensuring the suggestions are tailored to students' mistakes and learning needs, and are actionable; the final graded output format supports mainstream formats such as PDF, web version, and Word, based on the principle of adapting to the viewing needs of different users, with an output delay ≤ 1 second.

[0071] Structured report data refers to homework correction report data with standardized format and structured content. It includes answer overview (score, accuracy rate, answer time), score statistics (score for each question type, details of deductions), details of wrong questions (wrong question, wrong answer, correct answer), error attribution (wrong reason, summary of attribution), similar cases, analysis process, etc. The data format is standardized JSON format, and the values ​​are selected based on the following criteria: ensuring that the report data format is consistent, which is convenient for the feedback module to compile and process. The data integrity is ≥99.5%, ensuring that there is no missing content.

[0072] The feedback module is a functional module used to compile structured report data, generate intuitive feedback reports, and incorporate personalized improvement suggestions. It supports compilation output in multiple report formats, with a compilation delay of ≤1s. The criteria for this value are: ensuring fast compilation completion and improving output efficiency; compilation accuracy ≥99% to ensure that the report format is standardized and the content is error-free; and module operation stability ≥99.9% to avoid failures during the compilation process.

[0073] Compiling structured report data refers to the process by which the feedback module converts standardized structured report data into an intuitive, easy-to-understand, and viewable report format (such as PDF, web version, Word). The compiled report has a resolution of ≥1200dpi, which is determined based on the following criteria: ensuring that the report content is clear and easy for students and teachers to view; and a compilation error of ≤0.5%, ensuring that the report content is consistent with the structured report data without deviation.

[0074] Personalized improvement suggestions refer to targeted improvement suggestions tailored to students based on their error attribution, learning situation, and similar cases. These suggestions include knowledge point review suggestions (such as focusing on reviewing a specific knowledge point), problem-solving method guidance (such as optimizing problem-solving steps and mastering graphic analysis techniques), and writing standardization suggestions (such as improving illegible handwriting and reducing corrections). The relevance of the suggestions should be ≥98%, and the criteria for selection are: ensuring that the suggestions can accurately solve the student's errors and are actionable; the number of suggestions for each error should be 1 to 2, with 1 being preferred, and the criteria for selection are: avoiding too many suggestions, highlighting key points, and facilitating student implementation.

[0075] The final grading output refers to the complete homework grading result obtained after the feedback module compiles and incorporates personalized improvement suggestions. It includes a structured grading report, score, details of wrong answers, cause of wrong answers, similar cases, personalized improvement suggestions, etc. The output format supports mainstream formats such as PDF, web version, and Word. The selection criteria are: to adapt to the viewing and usage needs of different users (students and teachers); the accuracy rate of the output results is ≥99%, ensuring that the grading results are accurate and reliable, and can be directly used for student review and teacher teaching reference.

[0076] Furthermore, the intelligent job grading method based on multimodal recognition provided in this embodiment includes step S100 as follows: Step S110: Acquire educational image data through the image acquisition module, identify the text and graphic regions, and extract mixed text and graphic question data containing text descriptions and geometric figures.

[0077] Educational image data is acquired through an image acquisition module, such as scanning math exam papers with a high-resolution camera to identify text and graphic regions. Specifically, the text region contains question descriptions such as "find the area of ​​a triangle," while the graphic region contains the corresponding geometric figures. Data of mixed text-image questions containing both text descriptions and geometric figures is extracted, thus providing a foundation for subsequent analysis.

[0078] Step S120: For mixed text and image question data, use a semantic mapping mechanism to extract keyword vectors from the text content and simultaneously obtain visual feature vectors from the image region.

[0079] For the aforementioned mixed text and image question data, a semantic mapping mechanism is used to extract keyword vectors from the text content. This semantic mapping mechanism is based on a natural language processing model such as BERT, which decomposes the text into words and maps them to a high-dimensional vector space. For example, "triangle" is mapped to vector [0.2, 0.5, -0.1...]. Simultaneously, visual feature vectors of the graphic region are obtained, and edge and shape features, such as the vertex coordinate vectors of the triangle, are extracted from the graphic through a convolutional neural network such as ResNet, thereby capturing the visual essence.

[0080] Step S130: Based on the keyword vector and visual feature vector, analyze the spatial layout relationship between text and graphics on the page, and determine the positional offset between them.

[0081] Based on the keyword vector and the visual feature vector, the spatial layout relationship between text and graphics on the page is analyzed to determine the positional offset between them. Specifically, the center coordinates of the text area (e.g., (100, 200)) and the center coordinates of the graphic area (e.g., (150, 220)) are first calculated, and then the offset is calculated using the Euclidean distance formula, for example... Pixels; this offset reflects whether the text description is adjacent to the graphic. A small offset means that the text directly labels the graphic, thus ensuring consistent layout logic.

[0082] Step S140: Using a preset correlation strength calculation method, the position offset and the logical consistency of the knowledge test points are weighted and calculated to obtain the correlation strength that reflects the degree of fit between the text and the image.

[0083] The correlation strength is calculated using the following formula: (1) In formula (1), The strength of the association between the image and text is dimensionless and is calculated by real-time weighted fusion. Its value range is [value range missing]. Defined as the degree of close association between the title text and the corresponding graphic; This is a spatial location weighting coefficient, dimensionless, derived from system preset configuration parameters, with a value range of [value range missing]. , defined as the contribution weight of the positional relationship between images and text to the degree of relevance; This is the logical weighting coefficient for the test points, dimensionless, derived from system preset configuration parameters, and its value range is [value range missing]. , defined as the contribution weight of test point consistency to relevance; This is a normalized value for the text / image position offset, dimensionless, derived from the layout analysis module, and its value range is [value range missing]. , is defined as the normalized result of the offset between the center of the text and the center of the graphic region; To ensure logical consistency of the test points, the value is dimensionless, derived from the knowledge matching module, and its range is [range missing]. , defined as the degree of consistency between text semantics and graphic representation of test points. The control logic of formula (1) is to achieve a quantitative representation of the correlation between text and graphics by weighting and fusing indicators from two dimensions: text-graphic spatial layout and test point logic. The term converts the offset into positional fit, along with the weights. Multiplication reflects the positive contribution of spatial layout to the degree of interconnectivity; Multiplying by the weight β reflects the positive contribution of logical matching of test points to the relevance. The final output... The larger the value, the higher the degree of dependence of the question on the image. This formula (1) is the first to jointly calculate the image-text association strength by weighting two independent dimensions: image-text spatial layout offset and semantic logic consistency. It breaks through the limitations of traditional judgment based on a single dimension and achieves accurate identification of image-text coupled question types. This indicator can serve as the core basis for judging whether a question is a visually dependent question, providing data support for subsequent differentiated processing strategies (such as OCR enhancement and image parsing priority), and effectively improving the processing accuracy of complex image-text mixed questions.

[0084] Using a pre-defined method for calculating association strength, the positional offset is weighted and calculated based on the logical consistency of the knowledge points to obtain an association strength reflecting the degree of fit between the text and the image. Specifically, the logical consistency of the knowledge points is evaluated using a pre-trained knowledge graph. For example, the matching degree between the knowledge point "geometric area" and the text keywords is 0.8, and the matching degree with the graphic features is 0.9. The weighted formula is then applied as: Strength = 0.6. (1 / offset) + 0.4 The consistency value, if the offset is 53.85, then 1 / offset is approximately 0.019, and the weighted strength is approximately 0.72, thus quantifying the relationship between the image and text.

[0085] Step S150: If the correlation strength exceeds the preset logical threshold, the text semantics and graphic features are deeply aggregated through the feature fusion algorithm to finally determine the coupling degree value of the mixed text and graphic question type data.

[0086] The coupling degree value is obtained by the following formula: (2) In formula (2), This is the image-text feature coupling degree value, dimensionless, calculated in real-time by cosine similarity calculation, and its value range is normalized. , defined as the degree of fusion and matching between the semantic features of the title text and the visual features of the graphics; This is a text keyword feature vector, dimensionless, derived from the text semantic extraction module, and defined as a semantic feature vector generated by parsing the title text. The graphic visual feature vector is dimensionless and is extracted from a CNN convolutional neural network. It is defined as the visual feature vector generated by the analysis of the graphic in the question. This is a vector dot product operation used to calculate the inner product similarity of feature vectors; The L2 norm product of vectors is used to normalize the feature vectors. The control logic of formula (2) is to calculate the directional consistency of text and graphic feature vectors through the cosine similarity algorithm, and map the similarity between high-dimensional feature vectors to a scalar coupling degree value in the range of 0 to 1. When the coupling degree exceeds the preset threshold, the question is determined to be a visually dependent question, and the graphic analysis strategy should be used first for processing. Formula (2) uses cosine similarity to quantify the high-dimensional matching relationship between text and graphic features, breaking through the limitations of traditional rule matching alone, and realizing the accurate quantification of text-graphic coupling degree. This indicator provides a reliable classification basis for the automatic grading of multimodal questions, and can drive the system to automatically allocate graphic analysis resources for questions with high coupling degree, significantly improving the recognition and grading accuracy of complex text-graphic mixed questions.

[0087] If the correlation strength exceeds a preset logical threshold, such as 0.7, a feature fusion algorithm is used to deeply aggregate the text semantics and the graphic features to ultimately determine the coupling degree value of the mixed text-image question data. Specifically, the feature fusion algorithm employs an attention mechanism, concatenating keyword vectors and visual vectors and fusing them through a multilayer perceptron. For example, the input is a concatenated vector [keyword vector; visual vector], the output is a fused vector, and the coupling degree is calculated, such as a cosine similarity of 0.85. This aggregation improves the completeness of the question data, thereby optimizing the accuracy of automatic evaluation in the intelligent homework grading system. Through the above methods, efficient text-image analysis is achieved.

[0088] Preferably, the intelligent job grading method based on multimodal recognition provided in this embodiment includes step S200 as follows: Step S210: If the coupling degree value exceeds the preset threshold, the text-image mixed question type data is determined to be a visually dependent question.

[0089] When the coupling degree exceeds a preset threshold such as 0.8, the intelligent homework grading system determines that the mixed text and image question data is a visually dependent question. For example, in educational test papers, math questions containing complex geometric figures are identified as highly dependent on visual elements, and thus enter the further processing flow.

[0090] Step S220: Extract graphic elements containing geometric topology and pixel distribution information from visually dependent questions.

[0091] Extracting graphic elements from visually dependent questions begins with identifying geometric topological information using edge detection algorithms such as the Canny operator. For example, in a triangle area problem, the topological structure formed by vertices and edges is detected, with vertices acting as nodes and edges representing connections. Simultaneously, pixel distribution information is analyzed, using histograms to statistically analyze pixel intensity distribution. For instance, if grayscale values ​​within the graphic area are concentrated in the 100-150 range, it reflects the ink density, thus obtaining a complete description of the graphic element. This extraction process ensures the accuracy of subsequent analysis because geometric topology captures an abstract representation of the shape, while pixel distribution provides low-level visual cues. For example, in practical applications, for a scanned physics exam paper, the intelligent homework grading system first binarizes the graphic area to filter out background noise, then calculates connected components to divide independent elements, such as separating multiple geometric shapes, ensuring that each element contains independent topological and pixel data.

[0092] Step S230: Call the visual recognition module for the graphic element and use image processing algorithms to separate the handwriting strokes from the original graphic outline.

[0093] The handwriting difference image is obtained using the following formula: (3) In formula (3), The handwriting difference image matrix is ​​dimensionless and is solved in real time by pixel difference operation. It is defined as the difference image matrix between student handwriting and standard printed graphics. The original image matrix of the assignment is dimensionless and comes from the front-end image acquisition module. It is defined as the original pixel matrix of the assignment containing printed graphics and handwritten handwriting. The standard template image matrix for the question is dimensionless and comes from the standard graphic database of the question bank. It is defined as the pixel matrix of the unwritten reference template of the standard printed graphic of the question. The absolute value of the pixel difference is calculated by subtracting each pixel and taking the absolute value to eliminate the influence of positive and negative signs, reflecting only the magnitude of the difference. The control logic of formula (3) is to perform grayscale value difference operation on each pixel between the original acquired image and the standard template image, and then eliminate the influence of positive and negative signs through the absolute value operator to generate a difference image matrix. In the difference image, the pixel difference of the standard printed graphic area is close to 0, while the pixel difference of the student's handwritten handwriting area is significant, thus achieving accurate separation of handwritten handwriting and original printed graphics. Formula (3) uses the pixel difference method to quickly separate printed graphics and handwritten handwriting, without the need for a complex semantic segmentation model, with high computational efficiency, and is suitable for grading scenarios of various question types such as geometry questions, drawing questions, and calculation questions; it solves the pain point of difficulty in distinguishing handwriting from background graphics in traditional visual recognition, and provides pure handwriting feature data support for subsequent handwriting recognition, answer content analysis, and grading, and improves the digital processing link of the entire process of grading mixed text and graphics questions.

[0094] For the extracted graphic elements, the visual recognition module is invoked. This module is implemented based on the OpenCV framework and uses image processing algorithms such as morphological operations to separate handwriting strokes from the original graphic contours. For example, dilation and erosion operations are used to remove minor noise. Then, contour tracking algorithms such as the Suzuki algorithm are applied to extract contour lines, distinguishing handwritten strokes, such as auxiliary lines marked by students, from original printed graphics, such as standard triangles. Specifically, handwriting strokes usually have irregular width variations, while the original contours are smoother, thus achieving separation through threshold filtering.

[0095] Step S240: Perform handwriting analysis on the strokes and perform graphic analysis on the original graphic outline to extract structural nodes and stroke trajectories.

[0096] Handwriting analysis involves trajectory reconstruction algorithms, such as using the least squares method to fit the stroke curve and extract features such as the starting point, direction vector, and curvature. At the same time, the original graphic outline is analyzed, and straight lines or arcs are detected by Hough transform. Structural nodes such as intersection coordinates and stroke trajectories such as continuous path sequences are extracted. For example, in geometry problems, the three vertices of a triangle are analyzed as nodes, and the trajectory path of the edge is traced.

[0097] Step S250: Summarize the structural nodes and stroke trajectories to finally determine the recognition feature set that reflects the characteristics of the question.

[0098] By summarizing structural nodes and stroke trajectories, a set of recognition features is finally determined. For example, node coordinates and trajectory vectors are concatenated into a multi-dimensional feature vector, which is used for question classification in the intelligent homework grading system, thereby improving the accuracy of automatic grading.

[0099] Furthermore, the intelligent job grading method based on multimodal recognition provided in this embodiment, S330 includes: Step S310: Search the preset educational ontology database according to the recognition feature set, construct the semantic association matrix by calculating the cosine similarity between the feature vector and the node label, and determine the knowledge graph nodes that are highly related to the content of the question.

[0100] The cosine similarity between the feature vector and the node label is calculated using the following formula: (4) In formula (4), The cosine similarity is dimensionless and is calculated in real-time using cosine similarity calculations. Its value range is... , defined as the matching degree between question recognition features and knowledge graph nodes; The feature set vector for the question is dimensionless and comes from the visual / text multimodal feature fusion module. It is defined as the comprehensive feature vector of the question. The knowledge graph node feature vector is dimensionless and comes from the pre-trained embedding of the educational ontology library. It is defined as the knowledge point feature vector. The L2 norm product of vectors is used to normalize the feature vectors. The control logic of formula (4) is to calculate the directional consistency between the comprehensive feature vector of the question and the knowledge node vector through the cosine similarity algorithm, and map the high-dimensional feature matching result to a scalar similarity value in the range of 0 to 1; according to the similarity sorted from high to low, a preset number of high similarity nodes are selected to construct the knowledge point association reasoning chain corresponding to the question. Formula (4) uses cosine similarity to realize the accurate mapping of multimodal features of the question to nodes of the educational knowledge graph, which breaks through the limitation of traditional keyword matching being easily interfered by synonyms and polysemous words, and significantly improves the matching accuracy of questions and knowledge points; it provides a reliable foundation for subsequent grading based on source tracing and knowledge point association reasoning, ensures the professionalism and accuracy of the grading process, and improves the knowledge support link for intelligent grading of multimodal questions.

[0101] The intelligent homework grading system first searches a pre-defined educational ontology database based on the identified feature set. This database is a structured knowledge storage system containing standardized concepts, relationships, and attributes from subjects such as mathematics and physics. For example, it stores definitions of geometric figures, such as the side-angle relationships of a triangle. A semantic association matrix is ​​constructed by calculating the cosine similarity between feature vectors and node labels. Specifically, the cosine similarity calculation involves representing the feature vector as a multi-dimensional numerical array, such as a vector containing vertex coordinates and trajectory vectors. This array is then multiplied by the embedding vector of the node label, and the result is a similarity value, forming a semantic association. The elements of the semantic association matrix represent the strength of the association between different features and knowledge nodes. For instance, when processing a problem about the area of ​​a triangle, the intelligent homework grading system retrieves nodes related to "base" and "height," ensuring that the semantic association matrix highlights these core elements.

[0102] Step S320: For the structured data stored in the knowledge graph nodes, extract the formula strings containing mathematical logic and descriptive text, use a syntax parser to construct a formula parse tree and generate the corresponding text semantic vector.

[0103] After retrieval, knowledge graph nodes highly relevant to the question content are identified. These nodes are the basic units in the knowledge graph, storing structured data such as formulas and text. The intelligent homework grading system extracts formula strings containing mathematical logic and descriptive text from them, and uses a syntax parser to construct a formula parse tree. Specifically, the syntax parser is a tool based on context-free grammars, which parses formula strings such as " "Decompose it into a tree structure, with the root node being the operator." The child nodes include the constant "1 / 2" and the variable " "and" At the same time, corresponding text semantic vectors are generated. Descriptive text such as "calculate the area of ​​a triangle" is converted into vector representations through word embedding models such as Word2Vec, thereby capturing semantic meaning.

[0104] Step S330: Input the formula parsing tree and text semantic vector into the subject logic engine, retrieve the matching problem-solving logic and judgment criteria in the pre-stored subject rule base, and form a logical reasoning chain that supports the grading logic.

[0105] The formula parsing tree and text semantic vector are input into the subject logic engine, which is a software module based on rule reasoning. The engine retrieves matching problem-solving logic and judgment criteria from a pre-stored subject rule base. The subject rule base contains predefined logic templates such as "If the base is known and the height is correctly labeled, then the area calculation is valid". Through the matching process, a logical reasoning chain supporting the grading logic is formed. For example, the engine traverses tree nodes and compares vector similarity to construct a chain such as a sequence from "identifying the base" to "verifying the height" and then to "applying the formula".

[0106] Step S340: Calculate the rule matching degree of each problem-solving link according to the logical reasoning chain, quantify the matching results using the preset scoring dimension weights, and finally summarize and generate a set of grading criteria that reflects the evaluation standards of the questions.

[0107] The overall rule matching degree is calculated using the following formula: (5) In formula (5), The total rule matching degree is dimensionless and is calculated in real time by multi-dimensional weighted summation. Its value range is... This is defined as the overall degree of conformity between the problem-solving process and the subject rules and scoring criteria. The total number of scoring dimensions is dimensionless and comes from the subject rule base and scoring standards. It is defined as the total number of scoring items such as problem-solving steps, logical deduction, and correctness of results. For the first The weights of the scoring dimensions are dimensionless, derived from the preset scoring criteria configuration, and their value range is [range missing]. , is defined as the weight coefficient of each scoring item; For the first The item's single-dimensional rule matching value is dimensionless, derived from the single-dimensional rule validation output, and its value range is [value range missing]. , is defined as the degree to which the problem-solving process conforms to the rules of the subject in the corresponding dimension. As a multi-dimensional weighted summation operator, the matching values ​​of each dimension are weighted and accumulated according to the weight to obtain the overall matching degree. The control logic of formula (5) is to traverse each scoring dimension and check the matching degree of the problem-solving process with the subject rules in that dimension; then, according to the preset weight, the matching values ​​of each dimension are weighted and accumulated to obtain the overall rule matching degree; based on this, a set of grading basis containing the scores, deduction points and rule basis of each dimension is generated. Formula (5) adopts a multi-dimensional rule weighted matching mechanism, which breaks through the limitations of the traditional method of judging scores based solely on the correctness of the results, and realizes the step-by-step and subject-specific accurate grading of the problem-solving process; through the weight configuration, it can be adapted to the scoring standards of different question types and different subjects. The generated set of grading basis not only contains the total score, but also clearly shows the scoring points and deduction points of each link, which greatly improves the professionalism, transparency and interpretability of the grading results, and provides accurate support for subsequent error attribution and teaching feedback.

[0108] Based on the logical reasoning chain, the rule matching degree of each problem-solving link is calculated. The matching results are quantified using preset scoring dimension weights, such as accuracy accounting for 60% and completeness accounting for 40%. Finally, a set of grading criteria reflecting the evaluation standards of the questions is generated, thereby completing the automatic evaluation.

[0109] Preferably, the intelligent job grading method based on multimodal recognition provided in this embodiment includes step S400 as follows: Step S410: Generate guided queries using a large language model for the set of criteria for correction. The guided queries contain semantically guided query sequences.

[0110] The intelligent homework grading system utilizes a large language model to generate guided queries based on the grading criteria set. For example, when processing math homework, the large language model analyzes the formula matching rules in the criteria set and generates a query sequence containing semantic guidance, such as the sequence "Please explain the relationship between the base and height in area calculation," thereby guiding subsequent evaluation. For instance, the large language model is a deep learning framework based on the Transformer architecture. It captures semantic associations through pre-training on massive amounts of text data. Specifically, when generating queries, the large language model inputs the text vectors of the grading criteria set into an attention mechanism. Through multi-head self-attention, it calculates the weight relationships between different words, forming context-related embedding representations. Then, a decoder is used to progressively construct the query sequence, where each query embeds semantic prompts such as "derive the steps based on known conditions" to ensure the logical progression of the sequence. For example, in geometry problems, the sequence progresses from "identify the type of shape" to "verify the calculation formula," thus forming a complete guided chain. This approach, through semantic vector similarity matching, avoids queries deviating from the topic and optimizes the targeting of the evaluation.

[0111] Step S420: Use a small model to batch process the guided query of the link and obtain the score value and error point set.

[0112] The guided query is processed using a small model for grading, resulting in a score and a set of errors. The small model is a streamlined neural network, such as a variant based on DistilBERT, which extracts knowledge from a large model through distillation techniques, reducing the number of parameters to improve efficiency. Specifically, the grading process involves inputting the query sequence into the input layer of the small model, which extracts features layer by layer and classifies them. For example, in the grading process, the query response is first tokenized, and then the probability distribution is calculated through forward propagation to output a score, such as 85 points, while simultaneously identifying a set of errors, such as "incorrect formula application," thereby quantifying the student's problem-solving performance.

[0113] Step S430: Determine whether the set of error points involves handwriting deviation. If it involves handwriting deviation, analyze the handwriting image using an image recognition algorithm to generate a deviation vector.

[0114] The handwriting deviation vector is derived using the following formula: (6) In formula (6), The deviation vector is generated in real time by image contour comparison and statistical analysis, and is defined as a four-dimensional vector representing the deviation of handwriting. for The horizontal offset component, in pixels, is derived from the comparison of the centroid coordinates of the handwriting and the standard template, and is defined as the horizontal pixel offset. for The vertical offset component, in pixels, is defined as the pixel offset in the vertical direction. The stroke angle deviation component, in rad, is calculated by fitting the stroke direction and is defined as the angle deviation value between the main direction of the handwriting and the standard stroke. The stroke thickness deviation component, measured in pixels, is obtained from stroke width statistics and is defined as the thickness deviation value between the average line width of the handwriting and the standard template. The matrix transpose operator is used to convert row vectors into column vectors for easier subsequent vector operations. The control logic of formula (6) extracts the deviation components of handwriting in four dimensions—horizontal position, vertical position, writing angle, and stroke thickness—through contour centroid comparison, principal direction fitting, and line width statistics, and combines them into a complete four-dimensional deviation vector. Subsequently, the handwriting image can be geometrically corrected and morphologically normalized based on this vector to eliminate recognition interference caused by non-standard writing. Formula (6) constructs a four-dimensional handwriting deviation vector, realizing a multi-dimensional and structured representation of writing deviations, breaking through the limitations of traditional correction based on a single indicator. Based on this vector, handwriting can be specifically corrected, effectively reducing recognition errors caused by illegible writing, tilted angles, and uneven thickness, significantly improving the system's robustness to non-standard handwriting, and providing more stable feature data support for subsequent correction.

[0115] Determine whether the set of error points involves handwriting deviation. If so, analyze the handwriting image using an image recognition algorithm to generate a deviation vector. The image recognition algorithm is a tool based on a convolutional neural network. It converts the handwriting image into a grayscale matrix, extracts edge features such as stroke curvature, and then calculates the difference from the standard template to form a deviation vector, such as a two-dimensional array representing the positional offset.

[0116] Step S440: Merge the deviation vector and the error point set to perform handwriting correction and obtain preliminary correction results.

[0117] By integrating the deviation vector with the set of error points for handwriting correction, preliminary correction results can be obtained. For example, a weighted average method can be used to integrate the vector and error points, thereby correcting calculation errors caused by blurry handwritten numbers, and ultimately improving the accuracy of correction.

[0118] Furthermore, the intelligent job grading method based on multimodal recognition provided in this embodiment includes step S500 as follows: Step S510: Extract the attribution factors of incorrect questions from the preliminary correction results, and map the attribution factors of incorrect questions to knowledge graph nodes through semantic feature analysis to determine the knowledge point sequence.

[0119] The system extracts attribution factors from the initial grading results. For example, the intelligent homework grading system first analyzes the structured data of the grading results, classifying errors such as calculation mistakes or conceptual confusion into attribution factors. These attribution factors are identified using keyword extraction algorithms, such as "misuse of formulas" corresponding to knowledge blind spots, thus forming a factor list for subsequent analysis. Semantic feature analysis maps these attribution factors to knowledge graph nodes to determine the knowledge point sequence. Semantic feature analysis is a tool based on natural language processing that converts factor text into vector representations. For example, using the Word2Vec model to calculate the cosine similarity between word vectors, "area calculation error" is mapped to the "geometric formula" node in the knowledge graph. A knowledge graph is a graph database structure where nodes represent knowledge points, such as "triangle area," and edges represent relationships, such as "depends on base and height." The mapping process involves querying the graph interface, inputting factor vectors, traversing adjacent nodes, and generating a knowledge point sequence, such as an ordered chain from "basic definition" to "application rules." This method ensures that the sequence captures the core path of the error.

[0120] Step S520: Match according to the knowledge point sequence scheduling cache mechanism to obtain a set of candidate similar cases.

[0121] Based on the knowledge point sequence scheduling caching mechanism, a candidate similar case set is obtained. The caching mechanism is a memory storage strategy that preloads historical case data to accelerate access. For example, after the knowledge point sequence is input, the intelligent homework correction system calls the LRU algorithm to retrieve matching items from the cache. If a match is found, it is returned directly; otherwise, it is retrieved from the database. If the candidate similar case set contains past homework cases with similar geometric errors, it provides a basis for extended analysis.

[0122] Step S530: Calculate the retrieval weight for the candidate similar case set. If the retrieval weight exceeds the preset threshold, extract supplementary similar cases.

[0123] The similar case retrieval weight is calculated using the following formula: (7) In formula (7), The retrieved weights are dimensionless and are calculated in real-time using the Sigmoid function mapping operation, with a value range of [value range missing]. , defined as the comprehensive matching confidence of incorrect questions and case studies; This is a similarity scaling factor, dimensionless, derived from system preset configuration parameters, and its value range is [value range missing]. , defined as the scaling factor for similarity mapping; The overall similarity between incorrect questions and case studies. =The natural constant, the exponential base of the Sigmoid function, realizes the nonlinear mapping from similarity to weight. The control logic of formula (7) is to nonlinearly map the linear similarity value to the retrieval weight in the range of 0~1 through the Sigmoid function, realize the normalization of similarity and the enhancement of confidence; when the weight exceeds the preset threshold, the case is judged as a high confidence reference case, which can be extracted as a supplementary basis for the attribution of wrong questions. Formula (7) uses the Sigmoid function to perform nonlinear normalization of similarity, and maps the matching degree of different dimensions into a weight value that can be directly used for screening, which solves the problem that it is difficult to set a uniform threshold when using similarity directly; through the scaling factor It allows for flexible adjustment of the sensitivity of weights to similarity, enabling highly accurate recommendations of similar cases, providing rich reference for attributing errors, and significantly enhancing the interpretability of grading results.

[0124] For the candidate similar case set, a retrieval weight is calculated. If the retrieval weight exceeds a preset threshold, supplementary similar cases are extracted. The retrieval weight calculation involves a weighted summation formula, for example, weight = similarity. Relevance is assessed by evaluating the intersection of cases using the Jaccard index. Relevance is based on sequence matching. If the relevance exceeds a threshold such as 0.8, more cases are extracted from an external library, such as variant geometry problems.

[0125] Step S540: By fusing similar cases and attribution factors for incorrect questions through association mapping, an extended attribution set is obtained.

[0126] By fusing the supplementary similar cases and the attribution factors for incorrect answers through association mapping, an expanded attribution set is obtained. For example, a graph embedding algorithm can be used to align case features with factors to form a fusion vector, thereby expanding the attribution set to include more comprehensive error causes. This fusion improves the comprehensiveness of attribution.

[0127] Preferably, the intelligent job grading method based on multimodal recognition provided in this embodiment includes step S600 as follows: Step S610: Based on the attribution dimensions and knowledge nodes in the extended attribution set, construct an initial data matrix containing knowledge sequences through semantic mapping.

[0128] Based on the attribution dimensions and knowledge nodes in the extended attribution set, an initial data matrix containing knowledge sequences is constructed through semantic mapping. For example, an intelligent homework grading system first identifies attribution dimensions such as "conceptual confusion" and knowledge nodes such as "algebraic equations." Semantic mapping is a vector space-based matching technique that converts dimensional text into embedded vectors and uses models such as BERT to extract semantic features, thereby mapping "variable substitution errors" to a chain in the knowledge sequence from "basic variable definitions" to "equation solution steps." This mapping process involves calculating vector similarity and traversing the knowledge graph to generate an initial data matrix, where rows represent dimensions and columns represent sequence nodes, forming a multidimensional array to capture a structured representation of the error path.

[0129] Step S620: The initial data matrix is ​​weighted using logical weights and mapped to a preset report structure template to obtain structured report data.

[0130] The initial data matrix is ​​weighted using logical weights and mapped to a preset report structure template to obtain structured report data. Specifically, logical weights are a rule-based assignment mechanism, such as assigning values ​​like importance scores to elements of the initial data matrix, using formulas like weight = relevance. Frequency, where the logical dependencies between relevance assessment nodes are used, and historical occurrence rates are statistically analyzed, are used to convert the initial data matrix into a weighted version. This weighted version is then mapped to report structure templates such as the "Error Analysis" and "Knowledge Review" sections, resulting in structured data that quantifies errors. This processing ensures the logical coherence of the report and highlights key attributions through weighting.

[0131] Step S630: Based on the structured report data, use the feedback module to compile the structured report data and incorporate personalized improvement suggestions to obtain the report content with integrated suggestions.

[0132] Based on the structured report data, a feedback module compiles the data and incorporates personalized improvement suggestions to obtain a report with integrated suggestions. For example, the feedback module is an integrated compiler that parses the structured report data and generates personalized improvement suggestions, such as suggesting "practice variable isolation techniques" for "equation errors." The compilation process involves feedback template filling and natural language generation, and the complete report is formed after incorporating the personalized improvement suggestions.

[0133] Step S640: Determine whether the content of the fusion suggestion report conforms to the logical association rules. If it conforms to the logical association rules, determine the final correction output.

[0134] Logical consistency in the report is determined by the following formula: (8) In formula (8), To ensure logical consistency in the grading report, dimensions are avoided, and the results are calculated in real-time from the item validity statistics, with a value range of [value range missing]. This is defined as the overall logical compliance ratio of the report content; The total number of report items, dimensionless, is derived from the report template structure definition and is defined as the total number of items such as knowledge points, error analysis, and improvement suggestions. For the first The report entry has logical validity, is dimensionless, and is output by a single logical rule. It takes the value 0 or 1, which means that the entry takes the value 1 if it conforms to the preset logical association rule, and takes the value 0 otherwise. The global arithmetic average operator is used to average the validity of all items to obtain the overall compliance ratio. The control logic of formula (8) is to traverse all items in the correction report, check whether they conform to the logical association rules between knowledge points, error points, and improvement suggestions, and mark the valid items; then calculate the proportion of valid items through the arithmetic average algorithm. When the overall compliance ratio exceeds the preset threshold, it is determined that the report is logically rigorous and without contradictions, and can be used as the final correction result output. Formula (8) uses the global arithmetic average method to quantitatively verify the logical consistency of the report, ensuring the logical rigor and readability between knowledge points, error analysis, and improvement suggestions in the correction report; it solves the problem that traditional automatic correction reports are prone to contradictions and mismatches between suggestions and error points, realizes closed-loop quality control of correction results, and ensures the accuracy and reference value of personalized improvement suggestions.

[0135] The system determines whether the content of the integrated suggestion report conforms to logical association rules. If it does, the final revise output is determined. Specifically, logical association rules are a verification framework, such as checking the causal consistency between suggestions and attributions. If the consistency is passed, a report is output, thereby improving the accuracy of the revise.

[0136] Please see Figure 2This invention provides an intelligent homework grading system based on multimodal recognition, used to implement the aforementioned intelligent homework grading method based on multimodal recognition. It includes a coupling degree value acquisition module 10, a feature set determination module 20, a grading basis set acquisition module 30, a preliminary grading result acquisition module 40, an extended attribution set acquisition module 50, and a final grading output determination module 60. The coupling degree value acquisition module 10 acquires mixed text and image question data from educational image data through an image acquisition module, processes the mixed text and image question data using a preset association strength calculation method, and obtains a coupling degree value. The feature set determination module 20 determines that if the coupling degree value exceeds a preset threshold, it is a visually dependent question. It extracts graphic elements from the determination result, calls a visual recognition module to fuse image processing algorithms for handwriting and graphic analysis, and determines the recognition feature set. The grading basis set acquisition module 30... The system comprises four modules: a preliminary grading result acquisition module 40, and an extended attribution set acquisition module 50. The former extracts attribution factors from the preliminary grading results, retrieves related knowledge graph nodes based on a feature set, processes the formulas and text content within the knowledge graph nodes using a text analysis algorithm to match subject rules, and obtains a grading basis set. The latter module 40 generates guiding queries based on the grading basis set using a large language model, processes these guiding queries using a small model grading link to calculate scores and error points, determines whether error points involve handwriting deviation, and if so, integrates handwriting correction to obtain preliminary grading results. The latter extracts attribution factors from the preliminary grading results, retrieves historical data using a caching mechanism to supplement similar cases, and obtains an extended attribution set. The final grading output determination module 60 generates structured report data based on the extended attribution set, compiles the structured report data using a feedback module, incorporates personalized improvement suggestions, and determines the final grading output.

[0137] The intelligent job grading method and system based on multimodal recognition provided in this embodiment achieves the following beneficial effects compared with the prior art: 1. It can accurately handle mixed text and image questions, with a coupling degree calculation accuracy of ≥99.3%. It can quickly distinguish visually dependent questions, and the graphic element recognition accuracy reaches 98.8%. It solves the problems of traditional grading being unable to adapt to mixed text and image questions and having large graphic recognition deviations, and is suitable for the grading needs of various subjects' text and image assignments.

[0138] 2. It integrates visual recognition and image processing algorithms, with a handwriting and graphic parsing latency of ≤300ms and an accuracy rate of ≥99% for feature set extraction. It can accurately capture graphic details and handwriting information, avoid correction errors caused by graphic recognition mistakes, and improve correction accuracy.

[0139] 3. Combining knowledge graphs and text analysis algorithms, the subject rule matching accuracy rate is ≥98.5%, and the compliance rate of the generated grading basis is 100%, solving the problems of rigid traditional grading rules and poor subject adaptability, and adapting to the grading of assignments in multiple subjects (mathematics, physics, biology, etc.).

[0140] 4. By adopting a collaborative grading model with different sizes, the accuracy rate of score calculation is ≥99.2%, the error recognition rate is ≥98.7%, and the accuracy rate of handwriting deviation correction is ≥99%, which effectively reduces misjudgments caused by blurry handwriting and non-standard writing. The initial grading accuracy rate is more than 40% higher than that of traditional methods.

[0141] 5. By using a caching mechanism to retrieve similar incorrect question cases, the response time for attribution supplementation is ≤500ms, and the completeness of the expanded attribution set is ≥98%, which can accurately locate the root cause of the incorrect question, avoid the one-sidedness of a single attribution, and provide accurate basis for personalized improvement.

[0142] 6. Generate structured correction reports, improving report generation efficiency by over 70%, with a personalized improvement suggestion adaptation rate of ≥98%. No need for manual compilation of incorrect questions and suggestions, significantly reducing teachers' correction workload (by over 65%), while helping students accurately identify and fill knowledge gaps.

[0143] 7. Fully automated grading process, improving grading efficiency by more than 80% compared to manual grading, with a single assignment grading time of ≤10 seconds, and supporting multiple question types and subjects. It is highly versatile and can be widely used in various educational scenarios such as primary and secondary schools and training institutions.

[0144] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for intelligent grading of assignments based on multimodal recognition, characterized in that, Includes the following steps: S100. Obtain mixed text and image question data from educational image data through the image acquisition module, process the mixed text and image question data using a preset correlation strength calculation method, and obtain the coupling degree value; S200. If the coupling value exceeds a preset threshold, it is determined to be a visually dependent question. Graphic elements are extracted from the judgment result. For the graphic elements, the visual recognition module is called to fuse image processing algorithms to perform handwriting and graphic analysis and determine the recognition feature set. S300. Obtain the associated knowledge graph nodes based on the recognition feature set, and use a text analysis algorithm to process the formulas and text content in the knowledge graph nodes to match subject rules, thereby obtaining a set of grading criteria. S400. For the set of correction criteria, a guiding query is generated through a large language model. The guiding query is processed by a small model correction link to calculate the score and error points. It is determined whether the error points involve handwriting deviation. If handwriting deviation is involved, handwriting correction is integrated to obtain the preliminary correction result. S500: Extract the attribution factors of wrong questions from the preliminary correction results, and use the scheduling and caching mechanism of the attribution factors of wrong questions to retrieve historical data to supplement similar cases, thereby obtaining an extended attribution set; S600. Generate structured report data based on the extended attribution set, compile the structured report data using the feedback module and incorporate personalized improvement suggestions, and determine the final graded output.

2. The intelligent job grading method based on multimodal recognition according to claim 1, characterized in that, Step S100 includes: S110. Acquire educational image data through the image acquisition module, identify the text and graphic regions within it, and extract mixed text and graphic question data containing text descriptions and geometric figures. S120. For the text-image mixed question data, use a semantic mapping mechanism to extract keyword vectors of the text content and simultaneously obtain visual feature vectors of the graphic area. S130. Based on the keyword vector and the visual feature vector, analyze the spatial layout relationship between text and graphics on the page, and determine the positional offset between them. S140. Using a preset correlation strength calculation method, the position offset and the logical consistency of the knowledge test points are weighted and calculated to obtain the correlation strength that reflects the degree of fit between the text and the image. S150. If the correlation strength exceeds a preset logical threshold, the text semantics and the graphic features are deeply aggregated through a feature fusion algorithm to finally determine the coupling degree value of the mixed text and graphic question type data.

3. The intelligent job grading method based on multimodal recognition according to claim 2, characterized in that, Step S200 includes: S210. If the coupling degree value exceeds the preset threshold, the image and text mixed question data is determined to be a visually dependent question. S220. Extract graphic elements containing geometric topology and pixel distribution information from the visually dependent questions; S230. The visual recognition module is invoked for the graphic element, and the image processing algorithm is used to separate the handwriting strokes from the original graphic outline. S240. Perform handwriting analysis on the handwriting strokes and perform graphic analysis on the original graphic outline to extract structural nodes and stroke trajectories. S250. Summarize the structural nodes and the stroke trajectory to finally determine the recognition feature set that reflects the characteristics of the question.

4. The intelligent job grading method based on multimodal recognition according to claim 1, characterized in that, Step S300 includes: S310. Based on the identified feature set, a search is performed in the preset educational ontology database. A semantic association matrix is ​​constructed by calculating the cosine similarity between the feature vector and the node label to determine the knowledge graph nodes that are highly related to the content of the question. S320. For the structured data stored in the knowledge graph nodes, extract the formula strings containing mathematical logic and descriptive text, construct the formula parse tree using a syntax parser, and generate the corresponding text semantic vector. S330. Input the formula parsing tree and the text semantic vector into the subject logic engine, retrieve the matching problem-solving logic and judgment criteria in the pre-stored subject rule base, and form a logical reasoning chain that supports the grading logic. S340. Calculate the rule matching degree of each problem-solving link according to the logical reasoning chain, quantify the matching results using the preset scoring dimension weights, and finally summarize and generate a set of grading criteria that reflects the evaluation standards of the questions.

5. The intelligent job grading method based on multimodal recognition according to claim 4, characterized in that, Step S400 includes: S410. A large language model is used to generate a guided query for the set of correction criteria, the guided query containing a semantically guided query sequence; S420. The guided query is processed using a small-scale model to obtain the score and error point set. S430. Determine whether the set of error points involves handwriting deviation. If it involves handwriting deviation, analyze the handwriting image using an image recognition algorithm to generate a deviation vector. S440. Merge the deviation vector and the set of error points to perform handwriting correction and obtain preliminary correction results.

6. The intelligent job grading method based on multimodal recognition according to claim 1, characterized in that, Step S500 includes: S510. Extract the attribution factors of wrong questions from the preliminary correction results, and map the attribution factors of wrong questions to knowledge graph nodes through semantic feature analysis to determine the knowledge point sequence. S520. Based on the knowledge point sequence scheduling and caching mechanism, a candidate similar case set is obtained; S530. Calculate the retrieval weight for the candidate similar case set. If the retrieval weight exceeds a preset threshold, extract supplementary similar cases. S540. By fusing the supplementary similar cases and the attribution factors of the wrong questions through association mapping, an extended attribution set is obtained.

7. The intelligent job grading method based on multimodal recognition according to claim 6, characterized in that, In step 530, the retrieval weight is obtained using the following formula: ； in, Weighting for similar cases in retrieval. This is the similarity scaling factor. The overall similarity between incorrect questions and case studies. It is a natural constant.

8. The intelligent job grading method based on multimodal recognition according to claim 1, characterized in that, Step S600 includes: S610. Based on the attribution dimensions and knowledge nodes in the extended attribution set, construct an initial data matrix containing knowledge sequences through semantic mapping. S620. The initial data matrix is ​​weighted using logical weights and mapped to a preset report structure template to obtain structured report data. S630. Based on the structured report data, the feedback module compiles the structured report data and incorporates personalized improvement suggestions to obtain the report content with integrated suggestions; S640. Determine whether the report content of the fusion suggestion conforms to the logical association rules. If it conforms to the logical association rules, determine the final correction output.

9. The intelligent job grading method based on multimodal recognition according to claim 8, characterized in that, Logical consistency in the report is determined by the following formula: ； in, To ensure logical consistency in the report corrections, The total number of report items, For the first Logical validity of each report item It is the global arithmetic mean operator.

10. A multimodal recognition-based intelligent job grading system, used to implement the multimodal recognition-based intelligent job grading method as described in any one of claims 1 to 9, characterized in that, include: The coupling degree value acquisition module is used to acquire mixed text and image question data from educational image data through the image acquisition module, process the mixed text and image question data using a preset association strength calculation method, and obtain the coupling degree value; The feature set determination module is used to determine a visually dependent question if the coupling value exceeds a preset threshold, extract graphic elements from the judgment result, and call the visual recognition module to fuse image processing algorithms to analyze handwriting and graphics for the graphic elements to determine the feature set. The grading basis set acquisition module is used to acquire associated knowledge graph nodes based on the recognition feature set, and to process the formulas and text content in the knowledge graph nodes using a text analysis algorithm to match subject rules, thereby obtaining the grading basis set. The preliminary correction result acquisition module is used to generate guiding queries for the correction basis set through a large language model, process the guiding queries using a small model correction link to calculate scores and error points, determine whether the error points involve handwriting deviation, and if handwriting deviation is involved, integrate handwriting correction to obtain preliminary correction results. The extended attribution set acquisition module is used to extract the attribution factors of wrong questions from the preliminary correction results, and to retrieve historical data to supplement similar cases based on the scheduling cache mechanism for the attribution factors of wrong questions, thereby obtaining the extended attribution set. The final correction output determination module is used to generate structured report data based on the extended attribution set, compile the structured report data using the feedback module and incorporate personalized improvement suggestions to determine the final correction output.

Images