Intelligent question analysis and calibration method based on multi-modal large model

Abstract

The invention discloses an intelligent question analysis and calibration method based on a multi-modal large model, and relates to the technical field of image processing, and the method mainly comprises the steps: firstly, carrying out the OCR processing of an input multi-modal question picture, and generating structured OCR data containing coordinates, types and subject attributes through a general OCR engine in combination with a subject exclusive recognition plug-in; performing question region identification and segmentation by taking the data as a drive, and establishing a correlation graph among a whole question region, an answering region and an auxiliary region; a reference answer matched with the question type is generated based on the OCR data; further generating deep answer analysis taking the OCR information as a tracing basis in combination with the OCR data and the reference answer; and finally, based on the analysis result, the answer and the OCR data, generating a quantitative scoring standard by taking the analysis integrity, the answer correctness and the data use accuracy as standards. According to the invention, full-automatic processing from multi-modal topic picture input to final quantitative scoring standard generation is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an intelligent problem parsing and calibration method based on a multimodal large model. Background Technology

[0002] In current intelligent question processing technologies in K-12 education, Optical Character Recognition (OCR) technology serves as a crucial entry point for extracting information from question images, and its accuracy and interoperability with subsequent processing are paramount. However, existing solutions generally suffer from a series of problems. First, traditional OCR engines are primarily designed for general documents and struggle to effectively adapt to complex images rich in specialized symbols, graphic annotations, and structured information found in various K-12 subjects, such as angle symbols in mathematical geometry diagrams or instrument labels in chemical experiment diagrams. This leads to the omission or misrecognition of such critical information during the recognition stage, causing subsequent processing to lack an accurate data foundation from the outset. Second, existing systems typically treat OCR as a separate preprocessing module, resulting in a severe disconnect between the recognized text information and subsequent steps such as question analysis, answer generation, and scoring. For example, OCR might successfully identify side length parameters in a geometric diagram, but the answer analysis steps may fail to effectively utilize these parameters, or the scoring criteria may fail to consider students' application of the annotation information in the diagram, causing logical breaks and information silos throughout the entire processing chain. Due to the fundamental problems of inaccurate source identification and insufficient process linkage, the existing technical system still relies heavily on manual intervention in the final evaluation and calibration stage. Teachers need to manually supplement the information missing by OCR, bridge the logic of text and graphics, and formulate scoring rules. This not only makes the calibration of the tens of thousands of question banks take several months, but also makes it difficult to guarantee the consistency of the final calibration results across different teachers due to differences in subjective judgment. This seriously restricts the efficiency and quality of K12 smart education applications. Summary of the Invention

[0003] To reduce the reliance on manual intervention in intelligent problem-solving technology and improve product efficiency and quality, this invention proposes an intelligent problem parsing and calibration method based on a multimodal large model, comprising the following steps: S1: By performing OCR processing on the input multimodal question images, the general OCR engine is used and subject-specific recognition plugins are called to recognize the text, symbols and annotations in the images, and structured OCR data containing image region coordinates, information type, subject attributes and content values ​​is generated. S2: Driven by structured OCR data, the question region is identified and segmented, the whole question region, the answer region and the auxiliary region are identified, and a regional association map is established to construct structured regional information; S3: Based on structured OCR data and structured regional information, a general answer generation framework is used in conjunction with a subject format adaptation plugin to generate reference answers that are compatible with multimodal question types; S4: Based on structured OCR data and reference answers, a general parsing framework is used in conjunction with subject-specific adaptation plugins to generate in-depth answer parsing with structured OCR data as the basis for tracing the parsing logic. S5: Based on in-depth answer analysis, reference answers, and structured OCR data, a quantitative scoring standard is generated with the completeness of analysis, the correctness of the answer, and the accuracy of data use as the evaluation criteria.

[0004] This invention constructs a closed-loop processing flow with subject-adaptive OCR as the data entry point and deep linkage between various modules, realizing full automation from input of multimodal question images to the generation of final quantitative scoring standards. It fundamentally solves the problems of inaccurate OCR recognition and disconnected process links in traditional methods, which lead to broken parsing logic and low calibration efficiency, thereby improving the accuracy, efficiency and standardization of question processing in K12 education.

[0005] Furthermore, in step S1, after generating the structured OCR data, the step further includes: verifying and supplementing the structured OCR data based on a subject knowledge graph.

[0006] Furthermore, in step S1, the subject-specific recognition plugin includes at least one of the following: a science plugin for recognizing science formula symbols and graphic annotations, a humanities plugin for recognizing text-image association annotations, and a language plugin for recognizing scene element annotations.

[0007] Furthermore, in step S2, the question area is identified in the following way: Text clustering is performed based on the coordinates of text blocks in structured OCR data. The proportion of printed / handwritten text blocks and subject feature labels are combined to distinguish and identify the whole question area, the answer area, and the auxiliary area.

[0008] Furthermore, in step S2, after completing the identification and segmentation of the question region, the following step is also included: The region segmentation results are validated based on subject-specific rules. If the validation fails, a re-detection is triggered or a manual intervention interface is provided.

[0009] Furthermore, in step S3, the reference answer includes the following forms: Generate a unique answer for objective questions, generate point-based or tiered answers for subjective questions, and generate formulaic answers for calculation or proof questions.

[0010] Furthermore, in step S3, the reference answer is generated in the following manner: The general answer generation framework determines the form of the reference answer based on the multimodal question type, and calls the corresponding subject format adaptation plugin to generate the reference answer in the appropriate form and format according to the subject attributes of the multimodal question.

[0011] Furthermore, step S3 is followed by the following step: Based on structured OCR data and subject knowledge graphs, the correctness of the generated reference answers is verified.

[0012] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention proposes an intelligent question parsing and calibration method based on a multimodal large model, which solves the problem of low recognition rate of traditional OCR for subject-specific symbols and graphic annotations by using subject-specific recognition plugins and subject knowledge graph verification. (2) With structured OCR data as the core driver, it ensures close linkage throughout the entire process from question area segmentation, reference answer generation to in-depth analysis and deepening, effectively eliminating logical breaks caused by information disconnection in the traditional process; (3) By using a general framework combined with subject-specific plugins, the generation of reference answers, analysis basis and scoring criteria is automated, and the output results are guaranteed to strictly conform to the norms and logic of each subject. Attached Figure Description

[0013] Figure 1 A flowchart illustrating the steps of an intelligent problem parsing and calibration method based on a multimodal large model; Figure 2 This is a modular architecture diagram of the calibration system. Detailed Implementation

[0014] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0015] This invention provides an intelligent question parsing and calibration method based on a multimodal large model. Its core lies in constructing a closed-loop system that uses high-precision, subject-adaptive optical character recognition (OCR) technology as its data foundation and automates the entire process of subsequent question parsing and evaluation calibration. The following complete, end-to-end embodiment describes how this method processes a multimodal question image containing complex text and image information. This embodiment clearly demonstrates each step from image input to the final generation of structured scoring criteria, showcasing how this invention systematically solves key problems in existing technologies, such as inaccurate OCR recognition, fragmented processes, and high reliance on manual intervention, through technology integration and process reengineering. Figure 1 As shown, this method includes the following steps: S1: By performing OCR processing on the input multimodal question images, the general OCR engine is used and subject-specific recognition plugins are called to recognize the text, symbols and annotations in the images, and structured OCR data containing image region coordinates, information type, subject attributes and content values ​​is generated. S2: Driven by structured OCR data, the question region is identified and segmented, the whole question region, the answer region and the auxiliary region are identified, and a regional association map is established to construct structured regional information; S3: Based on structured OCR data and structured regional information, a general answer generation framework is used in conjunction with a subject format adaptation plugin to generate reference answers that are compatible with multimodal question types; S4: Based on structured OCR data and reference answers, a general parsing framework is used in conjunction with subject-specific adaptation plugins to generate in-depth answer parsing with structured OCR data as the basis for tracing the parsing logic. S5: Based on in-depth answer analysis, reference answers, and structured OCR data, a quantitative scoring standard is generated with the completeness of analysis, the correctness of the answer, and the accuracy of data use as the evaluation criteria.

[0016] The following detailed description of the specific implementation of this invention is based on a typical K-12 multi-subject question processing scenario. First, the system receives multimodal question images uploaded by users via scanners, cameras, or electronic devices. These images may originate from printed materials, handwritten manuscripts, or screenshots, and their content covers various types, including mathematical geometry proofs, physics circuit analysis problems, Chinese reading comprehension questions with illustrations, chemical experiment diagrams, or English dialogue scenarios. Figure 2 As shown, the calibration system employing the technical solution of this invention first activates the multimodal image OCR processing module. The core task of this module is to transform unstructured image information into semantically rich structured data. General optimization operations are performed through the image preprocessing unit, including noise reduction using a Gaussian filtering algorithm, adjusting the image angle using Hough transform-based tilt correction technology, and improving the clarity of low-quality images using a super-resolution reconstruction model. The preprocessing process adaptively enhances the images according to the characteristics of different disciplines: for example, for mathematical geometry images, the system uses the Canny edge detection algorithm to strengthen line contours, ensuring clear geometric boundaries; for chemical experiment images, color space analysis is initiated to distinguish the color characteristics of different reagents; and for historical data images, a color restoration model is applied to virtually repair faded areas.

[0017] After preprocessing, the image data is fed into a subject-specific OCR recognition unit. This unit runs a general text recognition model as its foundation, typically built on a CRNN or Transformer architecture, responsible for extracting general text information from the image. Simultaneously, the calibration system dynamically loads the corresponding subject-specific recognition plugin based on the question's metadata or the subject type determined through prior analysis. When processing science questions, the science plugin is activated; it integrates a formula recognition model based on an attention mechanism, capable of accurately parsing mathematical expressions such as "a...". 2 +b 2 =c 2 This plugin can handle complex structures such as the physics formula "F=ma". It also includes a graphic annotation detector specifically designed to locate and identify key annotations in geometric figures like "∠C=90°", physical circuit diagrams like "R1=10Ω", or chemical instruments like "50mL beaker". It can further analyze the structural relationships within the graphics, such as determining whether circuit components are connected in series or parallel. When dealing with humanities questions, the plugin focuses on establishing connections between text and graphics. For example, it can identify descriptive text in illustrations for Chinese reading comprehension, connect historical maps to time and location markers corresponding to specific figures or events, and interpret contour lines in geographical maps and their corresponding topographical meanings. For language-related questions, the corresponding language plugin focuses on identifying scene elements, such as recognizing "supermarket" and "goods" from English dialogue images. Visual identifiers representing shopping scenarios, such as "shelves" and "cashiers," or annotations describing human actions extracted from Japanese graphic captions, are used. All identified information, whether general text, specialized formulas, or graphic annotations, is uniformly encoded to generate a structured "OCR result table" (i.e., structured OCR data). Each record in this table includes its coordinate position in the original image (e.g., top-left corner coordinates (x1, y1) and bottom-right corner coordinates (x2, y2)), information type (clearly distinguishing between ordinary text, mathematical formulas, graphic annotations, and structural relationships), subject attribute (e.g., mathematical geometry, chemical experiment), and specific text or symbol content (e.g., "∠C=90°"). This step transforms the original image, which was merely a pixel array, into structured data with rich contextual information that machines can understand and process.

[0018] To ensure the reliability of the OCR results, the calibration system then activates the OCR result verification and completion unit. This unit accesses a pre-built knowledge graph database covering various K-12 subjects. For example, when processing mathematical geometry diagrams, if the OCR recognizes side lengths "AC=3cm" and "BC=4cm", but the knowledge graph's common sense hints about right triangles should include information such as "∠C=90°" or "AB is the hypotenuse", and this information is not successfully recognized, the system will automatically complete this key annotation in the corresponding coordinate area based on logical reasoning. Conversely, if the physics formula "F=ma" is misrecognized as "F=na", the unique representation of Newton's second law in the knowledge graph will trigger a correction mechanism to automatically correct it.

[0019] After obtaining high-quality structured OCR data, the workflow enters the question region segmentation and structuring module based on the OCR results. The goal of this module is to divide the entire question image into functional regions and establish logical relationships between them. First, the region segmentation preprocessing unit begins its work, normalizing the coordinates to map the coordinates of all OCR recognition boxes to a standardized image pixel coordinate system, thus eliminating the scale effect caused by differences in image resolution. Next, text clustering is performed, clustering physically adjacent text into larger "text blocks" based on the coordinate distance between OCR entries (e.g., setting a threshold of less than 5 millimeters for horizontal or vertical spacing). Simultaneously, the ratio of printed to handwritten characters within each text block is analyzed. For example, if more than 90% of the characters in a text block are regular printed characters, it is marked as a "question stem candidate block"; conversely, if handwritten characters dominate, it is marked as a "answer candidate area." Furthermore, this unit extracts subject-specific feature markers from the OCR data, such as identifying features containing " " Blocks containing symbols such as “” are “mathematical formula blocks”, blocks containing long, continuous paragraphs of text are “language paragraph blocks”, blocks surrounded by closed curves and containing short text are identified as “English dialogue bubble blocks”, and blocks containing clusters of circuit component symbols are “physics experiment blocks”.

[0020] Subsequently, the core region detection unit employs a specially trained target detection model. Its training data largely originates from the labeled text blocks generated during the aforementioned OCR process, enabling the model to deeply understand the correspondence between OCR features and region types. In actual detection, the model comprehensively utilizes three judgment criteria: "OCR text type (proportion of printed / handwritten text)," "subject feature marking," and "relative positional relationship of different regions on the page," to achieve accurate detection of the entire question region and the answer region. This multi-dimensional judgment method solves the misjudgment problem caused by relying solely on physical spacing for segmentation. For example, it can accurately exclude the printed annotation "∠C=90°" which is part of the question stem in a mathematical geometry diagram from the answer region; at the same time, for multiple handwritten answer regions that are very close in physical position, it can correctly associate each answer region with its corresponding sub-question by identifying and matching the tiny printed question number in front of it (such as "(1)", "(2)"), thus avoiding confusion.

[0021] Next, the auxiliary region detection unit is responsible for identifying regions that aid in understanding the question and evaluating the answer. This includes labeled regions containing prompts such as "Unit: cm" and "Note: Results are rounded to one decimal place," as well as nested sub-question regions in Chinese reading comprehension or comprehensive math problems (achieved by identifying sub-question numbers such as "①" and "②" in the OCR data). This auxiliary information is crucial for ensuring the standardization of answers and the accuracy of scoring. Then, the region structuring and association unit integrates all the detected regions to construct a "region association map" with the "whole question region" as the root node. This map clearly defines which answer regions and auxiliary regions are included in the whole question region, ensuring that in subsequent processing, the question stem information, student answers, and related prompts can be processed as an organic whole, preventing information gaps. Finally, the region segmentation and verification unit automatically verifies the segmentation results based on preset subject rules. For example, the rules may require that "the whole question region of a math calculation problem must correspond to at least one answer region," or "the whole question region of a Chinese reading comprehension question must contain a main paragraph text OCR block." If the verification fails, the system will automatically attempt to expand the area boundary for re-identification, or mark the area as "pending review" and open an interface for teachers to manually fine-tune it in subsequent processes. These adjustment data are also recorded for continuous optimization of the OCR and area detection model.

[0022] The third stage of the process is the reference answer generation module based on question stem OCR. This module takes structured OCR data as its sole input, and its core is a general answer generation framework that calls different generation strategies based on the question type. This task is executed by the answer format adaptation unit. For objective questions, such as multiple choice or true / false questions, the framework generates a unique and definite answer, such as option "C" or a Boolean value "". During the generation process, it actively references OCR data for verification. For example, if the answer to a math multiple-choice question is "5cm," the generation logic must be able to trace back to the parameters "AC=3cm" and "BC=4cm" identified by the OCR. For subjective questions, such as reading comprehension or historical analysis questions, the framework generates point-by-point and layered answer key points, such as summarizing the main idea of ​​the article or analyzing the impact of an event. In this process, the generator explicitly uses image annotations identified by the OCR as support. For example, if the answer key points mention "a warm atmosphere," it is based on keywords such as "sunshine" and "smiling face" identified by the OCR in the image. For calculation or proof questions, the framework generates strict formulaic answers, such as the mathematical formula "..." "or physical formula" Every symbol and parameter in these formulas must strictly adhere to the results recognized by OCR. Regarding subject-specific adaptation, the system uses plugins to ensure that the answer format conforms to subject-specific standards; for example, chemistry answers correctly indicate the reaction condition "ignition," and geography answers include the unit "meter" for contour line values.

[0023] To ensure the reliability of the answers, the answer accuracy verification unit again uses the "subject knowledge graph" to logically verify the generated answers. For the above mathematical answer "5cm", the verification logic simulates the problem-solving process: confirming the existence of OCR parameters "3cm" and "4cm", and confirming (or completing the knowledge graph) the existence of angle "90°", then applying the Pythagorean theorem to calculate and verify whether the result is 5cm. If the verification fails, it will trigger the regeneration of the answer or mark it as abnormal. For questions with multiple reasonable answers (such as multi-angle interpretations of Chinese reading), the system will generate a "complete solution answer" set containing all mainstream solutions, and clearly mark the OCR information support relied upon by each solution.

[0024] Based on the reference answer, the answer analysis deepening module generates not only accurate but also easy-to-understand analytical text. The OCR data association explanation unit ensures that every step of the analysis process is verifiable. The analytical text clearly explains why a certain OCR information is used. For example, in a mathematical analysis, it states: "Based on the OCR-recognized label '∠C=90°' in the image, triangle ABC can be determined to be a right triangle, thus satisfying the application conditions of the Pythagorean theorem, which is a key prerequisite for solving for side length AB." In a humanities analysis, it explains: "The judgment that the answer 'expresses the author's homesickness' is partly based on the images of 'bright moon' and 'lone boat' identified by OCR from the accompanying image. These images are often associated with the theme of homesickness in ancient poetry." Next, the subject-based supplementary unit is responsible for embedding necessary subject knowledge to increase the depth of the analysis. It automatically supplements the analysis with knowledge points such as "definition and formula of Pythagorean theorem" in mathematics, "basic function and effect of metaphor" in language arts, and "historical significance of the Xinhai Revolution in the process of world modernization" in history, so that the analysis not only provides steps but also explains the underlying principles. In addition, the abnormal answer verification unit analyzes common student errors. For example, when the system detects a common mathematical error "7cm", the analysis will point out: "This error stems from not using the key OCR information '∠C=90°' in the image, mistakenly adding the two right-angled sides of the triangle simply (3cm+4cm=7cm), without applying the Pythagorean theorem to calculate the hypotenuse." For the language arts error "expressing anger", the analysis will refute: "This interpretation does not match the 'smiling facial expression' annotation recognized by OCR in the image, therefore it is inaccurate." This attribution analysis combined with OCR data greatly enhances the relevance and teaching value of the analysis.

[0025] All information is aggregated into the scoring criteria calibration generation module to output objective and quantifiable scoring rules. The scoring dimension extraction unit first extracts core evaluation indicators from the "deep analysis" steps and "OCR-related information." Common basic dimensions include "answer correctness" (the degree of matching between the final answer and the reference answer) and "step completeness" (whether all key reasoning or calculation steps in the analysis are covered). More distinctive is the OCR-related dimension, which focuses on "the accuracy of OCR information usage." For example, in science questions, it examines "whether the correct parameters recognized by OCR were substituted into the formula," and in humanities questions, it examines "whether the key image annotations extracted by OCR were appropriately cited in the discussion." At the same time, through subject-specific scoring plugins, subject-specific dimensions such as "formula writing standardization" in mathematics (whether the subscripts and square signs are correct), "logicality and organization of expression" in Chinese, and "reasonableness of the order of experimental steps" in chemistry are introduced.

[0026] The scoring weight allocation unit then assigns appropriate weights to each dimension based on its importance. In a typical configuration, the core "correctness of the answer" might account for 40% of the weight, "completeness of steps" reflecting process evaluation might account for 30%, and "accuracy of OCR information use" reflecting information extraction and application ability might also account for 30%. The specific weight allocation is adapted to the subject characteristics through plugins; for example, in science questions, the weight of the "formula / parameter dimension" will be significantly higher than in humanities questions. The error tolerance rule setting unit defines flexible scoring rules. It clarifies the situation of "partial scoring," such as "if a student correctly substitutes the OCR parameters but makes a final calculation error, then the full score of the 'OCR information use' dimension will be given, but the score of the 'correctness of the answer' dimension will be deducted." It also stipulates the principle of "no repeated deductions," such as "if a score has already been deducted in the 'subject standardization' dimension due to an error in writing formula symbols, then no further deductions will be made in the 'correctness of the answer' dimension due to the calculation error caused by the error."

[0027] Finally, the scoring criteria output unit integrates all rules to generate a clearly structured and highly operable "structured scoring table." This table details each scoring dimension, its corresponding score, specific scoring rules, deduction rules, and error tolerance instructions, and also indicates the specific OCR data used for the scoring. For example, a scoring rule might state: "'Accurate OCR parameter substitution', worth 2 points. Rule: Correctly using the OCR-recognized 'AC=3cm' and 'BC=4cm' to substitute into the Pythagorean theorem formula earns 2 points; failure to use or incorrect substitution of any parameter deducts 2 points." The generated scoring table format is also adapted to the grading habits of various subjects through plugins, such as a "step-by-step table" commonly used in mathematics and a "key point table" commonly used in language arts.

[0028] In summary, this invention proposes an intelligent question parsing and calibration method based on a multimodal large model. Regarding the accuracy of source data, it solves the problem of low recognition rate of subject-specific symbols and graphic annotations in traditional OCR by using subject-specific recognition plugins supplemented by subject knowledge graph verification. Secondly, in terms of process collaboration and automation, it uses structured OCR data as the core driver, ensuring close linkage throughout the entire process from question region segmentation and reference answer generation to in-depth analysis, effectively preventing logical breaks caused by information gaps in traditional processes. In particular, by utilizing a general framework combined with subject-adaptive plugins, it not only automates the generation of reference answers, analysis basis, and scoring criteria, but also ensures that the output strictly conforms to the norms and logic of each subject. Finally, this solution automatically generates quantitative scoring criteria based on the accuracy of parsing steps and OCR data usage, transforming the calibration work from a subjective judgment heavily reliant on human experience into an efficient and objective automated process. This allows for the calibration of a question bank of tens of thousands of items in a very short time and improves the consistency of calibration across disciplines and teachers.

[0029] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0030] Furthermore, in this invention, descriptions involving terms such as "first," "second," and "a" are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0031] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0032] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

Claims

1. A method for intelligent problem parsing and calibration based on a multimodal large model, characterized in that, Including the following steps: S1: By performing OCR processing on the input multimodal question images, the general OCR engine is used and subject-specific recognition plugins are called to recognize the text, symbols and annotations in the images, and structured OCR data containing image region coordinates, information type, subject attributes and content values ​​is generated. S2: Driven by structured OCR data, the question region is identified and segmented, the whole question region, the answer region and the auxiliary region are identified, and a regional association map is established to construct structured regional information; S3: Based on structured OCR data and structured regional information, a reference answer adapted to multimodal question types is generated using a general answer generation framework combined with a subject format adaptation plugin; S4: Based on structured OCR data and reference answers, a general parsing framework is used in conjunction with subject-specific adaptation plugins to generate in-depth answer parsing with structured OCR data as the basis for tracing the parsing logic. S5: Based on in-depth answer analysis, reference answers, and structured OCR data, a quantitative scoring standard is generated with the completeness of analysis, the correctness of the answer, and the accuracy of data use as the evaluation criteria.

2. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 1, characterized in that, In step S1, after generating structured OCR data, the step further includes: verifying and supplementing the structured OCR data based on a subject knowledge graph.

3. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 1, characterized in that, In step S1, the subject-specific recognition plugin includes at least one of the following: a science plugin for recognizing science formula symbols and graphic annotations, a humanities plugin for recognizing text-image association annotations, and a language plugin for recognizing scene element annotations.

4. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 1, characterized in that, In step S2, the question area is identified in the following way: Text clustering is performed based on the coordinates of text blocks in structured OCR data. The proportion of printed / handwritten text blocks and subject feature labels are combined to distinguish and identify the whole question area, the answer area, and the auxiliary area.

5. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 1, characterized in that, In step S2, after completing the identification and segmentation of the question region, the following step is also included: The region segmentation results are validated based on subject-specific rules. If the validation fails, a re-detection is triggered or a manual intervention interface is provided.

6. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 1, characterized in that, In step S3, the reference answer includes the following forms: Generate a unique answer for objective questions, generate point-based or tiered answers for subjective questions, and generate formulaic answers for calculation or proof questions.

7. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 6, characterized in that, In step S3, the reference answer is generated in the following way: The general answer generation framework determines the form of the reference answer based on the multimodal question type, and calls the corresponding subject format adaptation plugin to generate the reference answer in the appropriate form and format according to the subject attributes of the multimodal question.

8. The intelligent problem parsing and calibration method based on a multimodal large model as described in claim 1, characterized in that, The step S3 is followed by the following step: Based on structured OCR data and subject knowledge graphs, the correctness of the generated reference answers is verified.

Images