Cross-modal correction template generation method, device, equipment and product

Abstract

The application discloses a cross-modal correction template generation method, device and equipment and product, and relates to the technical field of document image recognition. The method comprises the following steps: performing hierarchical structure identification on an original page image, outputting a hierarchical test paper skeleton from a layout space structure, making a topic set independent of a text sequence and artificial calibration and completely enumerated, then mapping answer texts according to test paper topics in the skeleton, normalizing an answer area into a candidate answer box and performing coordinate verification, and then merging a topic hierarchical relationship, a mapping relationship and a target answer box into the skeleton according to the same test paper topic, so that the answer semantics and the answer area coordinates are hung on a unified primary key. The application solves the problems that the prior art lacks an authoritative topic skeleton independent of a text sequence, and the answer texts and the answer area coordinates lack a unified primary key, leading to the problems of missing topics, repeated topics and scoring mismatch, and improves the efficiency of cross-modal correction template generation.

Description

Technical Field

[0001] This application relates to the field of document image recognition technology, and in particular to methods, apparatus, equipment and products for generating cross-modal correction templates. Background Technology

[0002] The existing automatic grading of paper-based exam papers mainly follows two routes in template generation. One is the pre-set coordinate route, which pre-marks the coordinates of the answer sheet and the answer area, and completes the layout correction and image slicing based on the positioning marks after scanning. The other is the text matching route, which first outputs the text lines in the reading order by OCR, and then extracts the question number using regular expressions, and then includes the answer text after the question number into that question. In recent years, there have also been solutions that input the entire page of the exam paper and the standard answer together into the visual language model for direct scoring, or use semantic matching and step-by-step comparison for scoring.

[0003] However, the above solutions focus on the scoring itself. The template process lacks authoritative questions that are independent of the text order and model output. The question set is either subject to manual calibration and becomes invalid when the paper is changed, or it is subject to the linear reading order of OCR. When encountering double columns, page spans, duplicate question numbers, or answer strings, the answer is assigned to the wrong question number and cannot be audited. Moreover, the semantics of the answer and the coordinates of the answer area are generated independently and lack a unified primary key. Errors accumulate and are transmitted to the scoring and rendering, ultimately leading to missing questions, duplicate questions, mismatch of scoring tasks, and label offset.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method, apparatus, device and product for generating cross-modal grading templates, which aims to solve the technical problems of existing technologies that lack authoritative question skeletons independent of text order and lack a unified primary key between answer text and answer area coordinates, resulting in missing questions, duplicate questions and mismatched scoring.

[0006] To achieve the above objectives, this application proposes a method for generating cross-modal correction templates, the method comprising: Obtain the original page image and the standard answer page image of the target test paper, and perform hierarchical structure recognition on the original page image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton; Based on the test questions in the hierarchical test paper skeleton, the answer text in the standard answer page image is mapped to the hierarchical test paper skeleton to obtain the mapping relationship; The original page image is subjected to answer region detection to obtain answer box regions. The answer box regions are then normalized to obtain candidate answer boxes. The candidate answer boxes are modified to obtain the target answer boxes. Based on the test questions in the hierarchical test paper skeleton, the question hierarchy, mapping relationship and target answer boxes are merged into the hierarchical test paper skeleton to obtain a structured grading template.

[0007] In one embodiment, the step of performing hierarchical structure recognition on the original test paper page image to obtain the question hierarchy and the hierarchical test paper skeleton includes: The original page image is corrected to obtain the corrected original page image; Optical character recognition is performed on the corrected original test paper page image to obtain the original test paper text block. Based on the original test paper text block and the corrected original test paper page image, hierarchical structure recognition is performed to obtain the initial test paper skeleton and the question hierarchy relationship between questions and sub-questions in the initial test paper skeleton. The sub-question is the lower-level question node of the question and the sub-question is the question node with the lowest level. The initial test paper skeleton was subjected to a structural consistency check, and the check results were obtained. Based on the inspection results, the nodes that passed the inspection in the initial test paper skeleton are retained, and the nodes that failed the inspection are registered as test paper items to be corrected. The test paper items to be corrected are manually corrected to obtain the corrected nodes. The nodes that pass the inspection and the corrected nodes are summarized to obtain the candidate test paper skeleton. The candidate test paper skeleton is associated with the question hierarchy to obtain a hierarchical test paper skeleton.

[0008] In one embodiment, the step of mapping the answer text in the standard answer page image to the hierarchical test paper skeleton based on the test paper questions of the hierarchical test paper skeleton, to obtain the mapping relationship, includes: Optical character recognition is performed on the standard answer page image to obtain the answer text block, and the answer text in the answer text block is normalized to obtain the normalized answer text; The question number is parsed according to the question number hierarchy parsing rule set to obtain the question number corresponding to the answer text; Based on the test paper questions in the layered test paper skeleton, the question number is matched with the questions in the layered test paper skeleton, while keeping the question structure and question order of the layered test paper skeleton unchanged, to obtain the mapping relationship between the answer text and the question. The answer texts that do not correspond to questions in the hierarchical test paper skeleton are registered as answer items to be corrected, and no new questions are added to the hierarchical test paper skeleton based on the answer items to be corrected.

[0009] In one embodiment, the step of detecting the answer region in the original page image to obtain the answer box region, and normalizing the answer box region to obtain the candidate answer box includes: Based on the layered test paper skeleton, the answer area of ​​the original test paper page image is detected to obtain the initial answer box area and the questions corresponding to the initial answer box area; The layered test paper skeleton is subjected to question anchor frame detection. If a question anchor frame is detected, the initial answer frame area is replaced by the question anchor frame to obtain a replacement answer frame area. If no question anchor frame is detected, the initial answer frame area is used as the replacement answer frame area. Conflict resolution is performed on the replacement answer box area to obtain the answer box area; Based on the page width and page height corresponding to the original page image, the answer box area is normalized to obtain candidate answer boxes.

[0010] In one embodiment, the step of modifying the candidate answer box to obtain the target answer box includes: The coordinates of the candidate answer box are compared with the page boundaries of the original page image to obtain boundary information; The degradation information of the candidate answer box is obtained by comparing the coordinates of the upper left corner and the lower right corner of the candidate answer box. Based on the question attribution information of the original page image, the candidate answer boxes are sorted to obtain the sorting order information; Based on the out-of-bounds information, degradation information, and arrangement order information, a verification result is generated, and the candidate answer box is corrected according to the verification result to obtain the target answer box.

[0011] In one embodiment, the step of correcting the candidate answer box based on the verification result to obtain the target answer box includes: Based on the boundary information, the candidate answer boxes that cross the page boundary are cropped to obtain cropped candidate answer boxes; Based on the degradation information, the cropped candidate answer boxes are deleted to obtain the retained answer boxes; The reserved answer boxes are sorted according to the arrangement order information to obtain the target answer boxes; Based on the test questions in the hierarchical test paper skeleton, the hierarchical relationship, mapping relationship, and target answer box of the questions are merged into the corresponding questions in the hierarchical test paper skeleton to obtain a structured grading template.

[0012] In one embodiment, after the step of merging the question hierarchy, mapping relationship, and target answer box into the hierarchical test paper skeleton based on the test paper questions of the hierarchical test paper skeleton to obtain a structured grading template, the method further includes: Receive a save request for the structured correction template, and generate a backup of the historical version of the structured correction template based on the save request; The structured correction template is analyzed to obtain a summary of the template content. Extract the question scoring points from the structured grading template, and associate and store the question scoring points with the template content summary to obtain a scoring point cache. Upon receiving the grading request for the target test paper, a summary analysis is performed on the structured grading template to obtain a real-time template content summary; If the real-time template content summary is consistent with the template content summary, then the target test paper is scored based on the question scoring points extracted from the scoring point cache to obtain the scoring result.

[0013] Furthermore, to achieve the above objectives, this application also proposes a cross-modal correction template generation device, which includes: The recognition module is used to acquire the original page image of the target test paper and the standard answer page image, and to perform hierarchical structure recognition on the original page image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton. The mapping module is used to map the answer text in the standard answer page image to the hierarchical test paper skeleton based on the test paper questions of the hierarchical test paper skeleton, so as to obtain the mapping relationship; The detection module is used to detect the answering region of the original page image, obtain the answering box region, and perform normalization processing on the answering box region to obtain candidate answering boxes; The correction module is used to correct the candidate answer boxes to obtain the target answer boxes, and to merge the question hierarchy, mapping relationship and target answer boxes into the hierarchical test paper skeleton according to the test paper questions of the hierarchical test paper skeleton to obtain a structured grading template.

[0014] In addition, to achieve the above objectives, this application also proposes a cross-modal correction template generation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the cross-modal correction template generation method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the cross-modal correction template generation method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the cross-modal correction template generation method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes a method, apparatus, device, and product for generating cross-modal grading templates. The method involves acquiring original test paper images and standard answer page images of a target test paper; performing hierarchical structure recognition on the original test paper images to obtain the question hierarchy and hierarchical test paper skeleton; mapping the answer text in the standard answer page images to the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain a mapping relationship; detecting answer areas in the original test paper images to obtain answer box areas; normalizing the answer box areas to obtain candidate answer boxes; correcting the candidate answer boxes to obtain target answer boxes; and merging the question hierarchy, mapping relationship, and target answer boxes into the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain a structured grading template. Therefore, by performing hierarchical structure recognition on the original test paper image, a hierarchical test paper skeleton is generated from the page space structure. This allows the question set to be completely enumerated independently of the text order and manual labeling. Then, the test paper questions in the skeleton are used as the basis to map the answer text, and the answer area is normalized into candidate answer boxes and the coordinates are verified. Then, the question hierarchy, mapping relationship and target answer box are merged into the skeleton based on the same test paper questions. This makes the answer semantics and the answer area coordinates attached to the same primary key. This solves the problem that the existing technology lacks authoritative question skeletons that are independent of the text order and lacks a unified primary key for answer text and answer area coordinates, which leads to the problems of missing questions, duplicate questions and mismatched scoring. This improves the efficiency of cross-modal grading template generation. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the cross-modal correction template generation method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the cross-modal correction template generation method of this application; Figure 3 A simplified flowchart illustrating the cross-modal correction template generation method provided in Embodiment 2 of this application; Figure 4 This is a schematic diagram of the module structure of the cross-modal correction template generation device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the cross-modal correction template generation method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is as follows: Image correction is performed on the original test paper page image to obtain a corrected original test paper page image; optical character recognition is performed on the corrected original test paper page image to obtain original test paper text blocks; and hierarchical structure recognition is performed based on the original test paper text blocks and the corrected original test paper page image to obtain an initial test paper skeleton and the hierarchical relationship between questions and sub-questions in the initial test paper skeleton, wherein the sub-question is a lower-level question node of the question and the sub-question is the lowest-level question node; structural consistency is checked on the initial test paper skeleton to obtain a check result; according to the check result, nodes that pass the check in the initial test paper skeleton are retained, and nodes that fail the check are registered as test paper items to be corrected; manual correction is performed on the test paper items to be corrected to obtain corrected nodes; the nodes that pass the check and the corrected nodes are summarized to obtain a candidate test paper skeleton; the candidate test paper skeleton is associated with the question hierarchy relationship to obtain a hierarchical test paper skeleton. Optical character recognition (OCR) is performed on the standard answer page image to obtain answer text blocks. The answer text blocks are then normalized to obtain normalized answer text. Question number parsing is performed on the normalized answer text according to a question number hierarchy parsing rule set to obtain the question number corresponding to the answer text. Based on the questions in the hierarchical test paper skeleton, the question number is matched with the questions in the hierarchical test paper skeleton, while maintaining the question structure and order of the hierarchical test paper skeleton, to obtain the mapping relationship between the answer text and the question. Answer texts without corresponding questions in the hierarchical test paper skeleton are registered as answer items to be corrected, and no new questions are added to the hierarchical test paper skeleton based on these answer items. Based on the layered test paper skeleton, the original test paper page image is used to detect the answer area to obtain an initial answer box area and the questions corresponding to the initial answer box area; the layered test paper skeleton is used to detect question anchor frames. If a question anchor frame is detected, the initial answer box area is replaced by the question anchor frame to obtain a replacement answer box area. If no question anchor frame is detected, the initial answer box area is used as the replacement answer box area; the replacement answer box area is conflict resolved to obtain the answer box area; the answer box area is normalized according to the page width and page height corresponding to the original test paper page image to obtain candidate answer boxes. The coordinates of the candidate answer boxes are compared with the page boundaries of the original exam paper image to obtain boundary information; the coordinates of the top left and bottom right corners of the candidate answer boxes are compared to obtain degradation information; the candidate answer boxes are sorted according to the question attribution information of the original exam paper image to obtain arrangement order information; a verification result is generated based on the boundary information, degradation information, and arrangement order information, and the candidate answer boxes are corrected according to the verification result to obtain the target answer box.Based on the boundary information, candidate answer boxes that cross the page boundary are cropped to obtain cropped candidate answer boxes; based on the degradation information, the cropped candidate answer boxes are region-deleted to obtain retained answer boxes; the retained answer boxes are sorted according to the sorting order information to obtain target answer boxes. A save request for the structured grading template is received, and a historical version backup of the structured grading template is generated based on the save request; the structured grading template is analyzed to obtain a template content summary; the question scoring points of the structured grading template are extracted, and the question scoring points are associated with the template content summary for storage to obtain a scoring point cache; when a grading request for the target test paper is received, the structured grading template is analyzed to obtain a real-time template content summary; if the real-time template content summary matches the template content summary, the question scoring points are extracted based on the scoring point cache to grade the target test paper, resulting in a grading result. This invention addresses the problems of existing technologies lacking authoritative question frameworks independent of text order, and the lack of a unified primary key between answer text and answer area coordinates, leading to issues such as missed questions, duplicate questions, and mismatched scoring. It enables the generation of cross-modal grading templates, improving generation efficiency. Based on this invention, it addresses the real-world problems of template generation lacking authoritative question frameworks independent of text order and model output. Furthermore, question sets are either limited by manual calibration and become invalid upon changing test papers, or constrained by the linear reading order of OCR, resulting in answers being incorrectly assigned to incorrect question numbers and becoming unauditable when encountering double columns, page breaks, duplicate question numbers, or answer strings, leading to low efficiency. A cross-modal grading template generation method was designed, and its effectiveness was verified during the generation of cross-modal grading templates. Finally, the efficiency of cross-modal grading template generation using this method was significantly improved.

[0025] In this embodiment, for ease of description, the cross-modal modification template generation device will be used as the execution subject for the following description.

[0026] Due to limitations in existing document image recognition technology and natural language processing algorithms, the accuracy and efficiency of cross-modal grading template generation need improvement. One issue is the reliance on authoritative questions. Existing solutions focus on the scoring itself, lacking authoritative questions independent of text order and model output in the template generation process. This compromises the reliability of the question set, leading to decreased accuracy in cross-modal grading template generation. Another issue is text order dependency; the text matching path is constrained by OCR. In linear reading order, when encountering double columns, page breaks, duplicate question numbers, or answer strings, the answer is assigned to the wrong question number and cannot be audited. The accuracy of cross-modal grading template generation also decreases significantly. Furthermore, there is the issue of manual calibration. Pre-defined coordinate routes are used to pre-calibrate the answer sheet and answer area coordinates. After scanning, layout correction and image slicing are completed based on the positioning marks. However, this becomes invalid once the paper is changed, which also affects the efficiency of cross-modal grading template generation. Therefore, in the current intelligent grading scenario, there are also difficulties in aligning answer semantics with answer area coordinates. Because the two are generated independently and lack a unified primary key, errors accumulate and propagate to scoring and rendering, ultimately leading to missed questions, duplicate questions, mismatched scoring tasks, and marker offsets, thus reducing accuracy and efficiency.

[0027] This application provides a solution that performs hierarchical structure recognition on the original test paper page image, generates a hierarchical test paper skeleton from the page space structure, and completely enumerates the question set independently of the text order and manual labeling. Then, the test paper questions in the skeleton are used as the basis to map the answer text, normalize the answer area into candidate answer boxes and perform coordinate verification, and then merge the question hierarchy relationship, mapping relationship and target answer box into the skeleton based on the same test paper questions. This makes the answer semantics and the answer area coordinates attached to a unified primary key, which solves the problems of existing technologies lacking authoritative question skeletons independent of the text order and lacking a unified primary key for answer text and answer area coordinates, resulting in missing questions, duplicate questions and mismatched scoring, and improves the efficiency of cross-modal grading template generation.

[0028] Based on this, embodiments of this application provide a method for generating cross-modal correction templates, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the cross-modal correction template generation method of this application.

[0029] In this embodiment, the cross-modal correction template generation method includes steps S01 to S04: Step S01: Obtain the original page image of the target test paper and the standard answer page image, and perform hierarchical structure recognition on the original page image to obtain the question hierarchy and the hierarchical test paper skeleton. Before the implementation of this embodiment, it should be clear that in the existing grading template generation, the template process lacks authoritative question basis that is independent of the text order and model output. The question set is either subject to manual calibration and becomes invalid when the paper is changed, or it is subject to the linear reading order of OCR. When encountering double columns, page spans, duplicate question numbers, or answer strings, the answer is assigned to the wrong question number and cannot be audited.

[0030] Therefore, in order to solve the above problems, this embodiment, after obtaining the original page images and standard answer page images of the same set of test papers, archives them separately according to page order, paper orientation and resolution, and records the batch identifier to which the page belongs, so as to establish an index for subsequent skeleton caching.

[0031] Subsequently, the orientation and size of each page of the original test paper image were corrected and standardized to ensure that the coordinate system of each page in the same test paper was consistent. Then, optical character recognition was performed on the corrected original test paper page image to obtain the original test paper text block with page number. This original test paper text block is only used to assist in locating the question number and does not directly determine the order of the questions.

[0032] Based on this, the original text blocks and the corrected original page images are sent together to the layout visual inspection and skeleton extraction. The spatial distribution of question number text, score text, option marks and separators in the layout produces a three-layer node consisting of major questions, questions and sub-questions. Based on the aforementioned spatial aggregation results, parent-child relationships are established between the nodes to obtain the initial test paper skeleton that records the hierarchical relationship of the questions.

[0033] Subsequently, the initial test paper skeleton is checked for the uniqueness of the question number identifier and the parent-child relationship for each node. Nodes that pass the check are retained, and nodes that fail are registered as test paper items to be corrected, thus obtaining candidate test paper skeletons. Finally, the candidate test paper skeletons are associated with the question hierarchy in the order of major question, question, and sub-question, thus obtaining the hierarchical test paper skeleton that serves as the basis for all subsequent alignment and sorting.

[0034] Step S02: Based on the test questions in the layered test paper skeleton, map the answer text in the standard answer page image to the layered test paper skeleton to obtain the mapping relationship; Once the tiered test paper framework is determined, the question set and question order are fixed, and thereafter only answer text is allowed to be attached to existing questions.

[0035] First, optical character recognition is performed on the standard answer page image to obtain the answer text block. Then, the blank characters, full-width and half-width characters, punctuation marks, uppercase and lowercase letters, and common question number writing in the answer text are uniformly normalized. At the same time, the original text, normalized text, page number, and text block coordinates are preserved for traceability.

[0036] Subsequently, the normalized answer text is parsed segment by segment according to the question number hierarchy parsing rule set, and the consecutive question number range, parenthetical question number, sub-question sequence, multiple choice answer string and multiple blank cases are restored to the corresponding question number.

[0037] Then, using the test questions in the layered test paper skeleton as the only matching object, the parsed question numbers are matched one by one with the questions in the skeleton. During the matching process, the question structure and question order of the skeleton remain unchanged, thereby obtaining the mapping relationship between the answer text and the questions.

[0038] If the answer text for a question does not exist in the skeleton, it will be registered as an answer item to be corrected, and no new question will be added to the tiered test paper skeleton based on it.

[0039] Step S03: Detect the answer region in the original page image to obtain the answer box region, and normalize the answer box region to obtain candidate answer boxes; In extracting the answer area, the visual model is called page by page to identify the answer area within the question range allowed by the skeleton, based on the established hierarchical test paper skeleton. The correspondence between each question or sub-question and the answer box coordinates is output. When the original test paper's own option markers or answer area anchor boxes already exist in the skeleton, the anchor box coordinates are used to replace the detection boxes output by the model. After resolving the conflicts of boxes with inclusion or overlap, the answer box coordinates are normalized to the range of zero to one according to the page width and page height of the original test paper image, thus obtaining candidate answer boxes decoupled from the scanning resolution and paper orientation.

[0040] Subsequently, coordinate verification is performed on each candidate answer box. The coordinates of the page boundary and the candidate answer box are compared to obtain out-of-bounds information. The coordinates of the upper left corner and the lower right corner of the candidate answer box are compared to obtain degradation information. The multiple blanks under the same question are sorted according to the question belonging information to obtain the sorting order information. The three are combined to obtain the verification result.

[0041] Step S04: Modify the candidate answer box to obtain the target answer box, and according to the test questions of the hierarchical test paper skeleton, merge the question hierarchy relationship, mapping relationship and target answer box into the hierarchical test paper skeleton to obtain a structured grading template.

[0042] After obtaining the verification results, the candidate answer boxes that cross the page boundary are cropped according to the page boundary based on the out-of-bounds information, and the degenerate answer boxes with non-positive width or height are deleted according to the degradation information, so as to obtain the retained answer boxes.

[0043] Then, the reserved answer boxes are sorted from top to bottom and from left to right according to the arrangement order information, so that the order of multiple answer blanks under the same test paper question is consistent with the natural writing order, thus obtaining the target answer box.

[0044] Next, using the test questions in the hierarchical test paper skeleton as the primary key, the hierarchical relationship of the questions, the mapping relationship, and the target answer box are merged one by one into the corresponding test question node in the skeleton. The answer text is written into the correct answer field, the target answer box is written into the answer box field, and the missing fields remain editable. The merged results are then sorted by natural question number and checked for completeness.

[0045] Finally, a backup of the old version and a content summary are generated each time a file is saved, resulting in a structured grading template that can be reused by the grading module, grade report module, grading symbol rendering module, and printing module.

[0046] This solves the problems of existing technologies lacking authoritative question skeletons independent of text order, and the lack of a unified primary key between answer text and answer area coordinates, which leads to missing questions, duplicate questions, and mismatched scoring, thus improving the efficiency of cross-modal grading template generation.

[0047] Specifically, step S01 above, which involves performing hierarchical structure recognition on the original test paper image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton, includes: Step S011: Perform image correction on the original roll page image to obtain the corrected original roll page image; Step S012: Perform optical character recognition on the corrected original test paper page image to obtain the original test paper text block, and perform hierarchical structure recognition based on the original test paper text block and the corrected original test paper page image to obtain the initial test paper skeleton and the question hierarchy relationship between questions and sub-questions in the initial test paper skeleton. The sub-question is the lower-level question node of the question and the sub-question is the question node with the lowest level. Step S013: Perform a structural consistency check on the initial test paper skeleton and obtain the check results; Step S014: Based on the inspection results, retain the nodes that have passed the inspection in the initial test paper skeleton, and register the nodes that have not passed the inspection as test paper items to be corrected. Step S015: Manually correct the test paper items to be corrected to obtain the corrected nodes, and summarize the nodes that have passed the inspection and the corrected nodes to obtain the candidate test paper skeleton. Step S016: Associate the candidate test paper skeleton with the question hierarchy to obtain the hierarchical test paper skeleton.

[0048] When the original test paper page image set and the standard answer key page image set of the same test paper enter the processing flow, the page order, paper orientation, resolution, and batch identifier are recorded together. The input sources of the images include, but are not limited to, images returned from scanners, images from printers with scanning capabilities, PDF files, and photographs. In this embodiment, page preprocessing can be performed using any of the following methods: traditional image processing, OCR engine, or visual language model. After establishing a skeleton cache key for the original page according to batch identifier, the cached skeleton result can be directly reused when the same original page is called repeatedly within the same batch, avoiding repeated calls to the visual model, thereby reducing waiting time and redundant calculations.

[0049] Image orientation correction, size unification, page numbering, and optical character recognition are performed on both the original paper and the answer key to obtain a set of text blocks with page numbers. The optical character recognition results on the original paper side are only used to assist in locating question numbers, while the optical character recognition results on the answer key side are used to obtain the standard answer text and score hints. Neither recognition result is directly used as the basis for the final question order. The optical character recognition of the original paper, the optical character recognition of the answer key, and the extraction of the visual answer box page by page can be performed in parallel without blocking each other.

[0050] For each original test paper page, the visual language model is invoked to output a hierarchical test paper skeleton S. The skeleton contains three layers of nodes: section (major questions), question (question), and sub-question (sub-question). Each node contains at least the question number identifier qid, question number text, page number, question stem summary, question type, maximum score, and coordinates of the title or question stem candidate area.

[0051] Skeleton extraction, answer alignment, and answer box extraction can be completed by the same visual language model at once, or by a combination of OCR, rule engine, large text model, and dedicated detection model. The rule engine and model can verify each other.

[0052] Before merging the output structure results for each page by page number, perform two consistency checks: The uniqueness of the question number identifier qid and the parent-child relationship check are performed. That is, each sub-question must have one and only one question parent node. Nodes that pass the check are retained. Nodes that fail the check are not deleted. Instead, their question number identifier and hierarchical position in the skeleton are retained and a correction mark is added. They are registered as test paper items to be corrected. The reason for retaining them instead of deleting them is that deletion would shrink the question set and cause the answer text to lose its attachment object, which would violate the principle of skeleton authority and would ultimately lead to missing questions.

[0053] The initial test paper framework adopts three levels: test paper sections, questions, and sub-questions. Test paper sections correspond to sections, questions correspond to questions, and sub-questions correspond to sub-questions. Sub-questions are subordinate question nodes that belong to questions and are the lowest-level question nodes in the framework. No further subordinate question nodes are set under them. The question hierarchy is the hierarchical relationship between a question and its subordinate sub-questions.

[0054] Spatial aggregation is performed on the question number text, score text, option markers and separators on the same page. Based on vertical and horizontal projection, continuous spaces or horizontal lines are divided into one or more blank (answer blank) nodes. The acceptable answer container types are recorded for multiple-choice questions, fill-in-the-blank questions and subjective questions respectively, namely option type, fill-in-the-blank type and area type. For test papers with deeper levels, the question nodes of deeper levels are merged under the nearest parent question node to maintain the three-level structure.

[0055] The test paper items to be corrected are displayed in the proofreading interface with their question number identifier, hierarchical position, question stem summary and failure reason. The question number, question name, score, parent-child relationship and sub-question split are modified manually. After the modification is completed, the uniqueness check of the question number identifier and the parent-child relationship check are re-executed. The nodes that pass the review have the pending correction mark cleared and the corrected nodes are obtained.

[0056] The nodes that pass the inspection and the nodes that pass the correction are summarized to obtain the candidate test paper skeleton. It should be noted that the manual correction is to modify the nodes that already exist in the skeleton, rather than to create new questions based on the answer text. This boundary ensures that the set of questions in the skeleton is always determined by the original test paper layout.

[0057] The skeleton authority rule is established here: the skeleton S is the sole authoritative source for all subsequent alignments and sorting. Answer text can only be mapped to a qid that already exists in the skeleton. Any answer text that does not have a corresponding qid in the skeleton is registered as an answer item to be corrected, and the system must not create a new question based on it. This hard constraint ensures that the question set is completely enumerated by the page structure, fundamentally avoiding the omissions, duplicates, and mismatches caused by determining the scoring task based on the linear text order of optical character recognition.

[0058] Finally, according to the hierarchical relationship of the questions, the questions in the candidate test paper skeleton are associated with the sub-questions belonging to the same question in the order of test paper section, question, and sub-question, to obtain the hierarchical test paper skeleton.

[0059] More specifically, step S02 above, which maps the answer text in the standard answer page image to the layered test paper skeleton based on the test paper questions of the layered test paper skeleton, to obtain the mapping relationship, includes: Step S021: Perform optical character recognition on the standard answer page image to obtain the answer text block, and normalize the answer text in the answer text block to obtain the normalized answer text; Step S022: Parse the question number of the normalized answer text according to the question number hierarchy parsing rule set to obtain the question number corresponding to the answer text; Step S023: Based on the test questions of the layered test paper skeleton, match the question number with the questions in the layered test paper skeleton, and keep the question structure and question order of the layered test paper skeleton unchanged to obtain the mapping relationship between the answer text and the question. Step S024: Register the answer texts that do not correspond to questions in the layered test paper skeleton as answer items to be corrected, and do not add any new questions to the layered test paper skeleton based on the answer items to be corrected.

[0060] Normalization is performed on the optical character recognition text on the answer side. The objects of normalization include blank characters, full-width and half-width characters, punctuation marks, uppercase and lowercase letters, and common question number writing, such as “1、”, “1.”, “(1)”, “Question 1”, etc., which are all unified into the same form. During the normalization process, the original text, normalized text, page number and text block coordinates are retained for subsequent tracing.

[0061] Using the question number identifier qid as the primary key, the answer text is mapped to the skeleton S according to the following set of rules, with each rule applied in order of priority.

[0062] Rule 1 applies to consecutive question number ranges, such as the text "1~5: ACBDA" in multiple choice questions and "1~3: False (×) True (√) False (×)" in true / false questions. The text is expanded into multiple qid according to the question number range, and the corresponding characters or fragments in the answer string are assigned to each qid in sequence.

[0063] Rule 2 applies to parenthetical question numbers and text in the form of “(1)……(2)……”, which are mapped to the sub-questions under the current question.

[0064] Rule 3 applies to the sequence of sub-questions, that is, sub-questions (1) to (N) under the same question are mapped in the order of their appearance.

[0065] Rule 4 applies to multiple-choice question answer strings, dividing the continuous unseparated uppercase letter string into equal parts based on the number of questions or splitting it according to the number of questions in the skeleton.

[0066] Rule 5 applies to multi-blank splitting. When a qid corresponds to multiple blanks, an ordered list of blanks is generated, and the answer fragment corresponding to each blank is retained.

[0067] When an answer text block covers multiple qids, it is split according to the question number range and then written separately. When a qid appears multiple times in the answer, the first hit is retained and subsequent hits are registered as conflict items to be corrected and handed over to manual confirmation.

[0068] This generates an answer mapping M, where each item in M ​​is of the form M={qid, sub_answer_source,confidence}, where sub_answer_source records the source of the answer fragment, including the page number, text block index, and coordinates, and confidence is the matching confidence.

[0069] For `qid` values ​​that exist in the skeleton but lack corresponding content in the answer, retain an empty `correct_answer` field and do not delete the question. Besides using `qid` as the primary key, weighted matching based on question number text, question stem summary, page number proximity, and answer text similarity can be used as auxiliary alignment criteria. Conflicting items can be handled through manual verification or secondary model review. Answers that cannot be uniquely mapped are marked as items awaiting correction and new questions are not generated arbitrarily. For `qid` values ​​that exist in the skeleton but lack corresponding answers, retain an empty standard answer field, thereby ensuring the integrity of the question set is not compromised by missing answers.

[0070] Further, step S03 above, which involves detecting the answer region in the original page image to obtain the answer box region, and normalizing the answer box region to obtain candidate answer boxes, includes: Step S031: Detect the answer area of ​​the original test paper page image according to the layered test paper skeleton to obtain the initial answer box area and the questions corresponding to the initial answer box area. Step S032: Detect question anchor frames on the layered test paper skeleton. If a question anchor frame is detected, replace the initial answer frame area with the question anchor frame to obtain a replacement answer frame area. If no question anchor frame is detected, use the initial answer frame area as the replacement answer frame area. Step S033: Conflict resolution is performed on the replacement answer box area to obtain the answer box area; Step S034: Based on the page width and page height corresponding to the original page image, normalize the answer box area to obtain candidate answer boxes.

[0071] The visual model is invoked for each original test paper page, and the model is required to identify the answer area of ​​the student or answer only within the question range allowed by the skeleton S, and output the mapping B between qid or sub_qid and the answer box coordinates. Since the recognition range is limited by the skeleton, the model must not output questions outside the skeleton, thereby suppressing the omission and mismatch of question numbers from the source.

[0072] The anchor frame priority rule is as follows: if the skeleton already contains the original paper's built-in option markers or answer area anchor frames, then the anchor frame coordinates are used first. The detection boxes output by the model are only used as a supplement. For subjective questions, it is allowed to retain both the question number box and the answer box coordinates returned by the model at the same time, which are used to locate the question and render the grading symbols, respectively.

[0073] For boxes with an inclusion relationship, the outer box is retained as the answer area and the inner box is retained as a sub-space. For multiple boxes of the same question with an overlapping area exceeding the threshold, they are treated as the smallest bounding box after merging and registered as items to be corrected for manual confirmation. This completes the conflict resolution of replacing the answer box area.

[0074] The coordinates are normalized to the range of zero and one using the page width and height, forming normalized coordinates [x1, y1, x2, y2]. This decouples the template from the scanning resolution, paper orientation, and page size, allowing it to be reproduced under different scanning conditions and thus obtaining candidate answer boxes.

[0075] The coordinates of the answer box can also be absolute pixel coordinates, percentage coordinates, quadrilateral vertex coordinates, or page grid coordinates, as long as they are converted to a unified page normalized coordinate system and the same verification is performed before merging.

[0076] Furthermore, step S03 above, the step of modifying the candidate answer box to obtain the target answer box, includes: Step S035: Compare the coordinates of the candidate answer box with the page boundary of the original page image to obtain boundary information; Step S036: By comparing the coordinates of the upper left corner and the lower right corner of the candidate answer box, the degradation information of the candidate answer box is obtained; Step S037: Sort the candidate answer boxes according to the question attribution information of the original page image to obtain the sorting order information; Step S038: Generate a verification result based on the boundary information, degradation information and arrangement order information, and correct the candidate answer box according to the verification result to obtain the target answer box.

[0077] Perform four-coordinate validation on the candidate response box set B: The first item is amplitude limiting, which restricts the coordinate values ​​to the range of zero to one. By comparing the page boundaries of the original page image with the coordinates of the candidate answer box, out-of-bounds information is obtained.

[0078] The second step is degeneracy removal. By comparing the coordinates of the top left and bottom right corners of the candidate answer boxes, degenerate boxes with x2 not greater than x1 or y2 not greater than y1 are deleted to prevent zero-area or inverted boxes from entering the subsequent scoring and rendering stages, thus obtaining degeneracy information.

[0079] The third item is cropping, which crops boxes that extend beyond the page boundaries according to the boundaries and records the cropping marks.

[0080] The fourth item is sorting. Based on the title attribution information in the original page image, multiple blanks with the same qid are sorted from top to bottom and from left to right to make their order consistent with the natural writing order, thus obtaining the arrangement order information.

[0081] The above-mentioned out-of-bounds information, degradation information and arrangement order information are combined to generate a verification result, which can be directly used for subsequent correction of candidate answer boxes. In this embodiment, the processing of the four-level verification can be trimmed or the order adjusted as needed. The sorting within the question can also be configured from right to left or according to the reading order template to adapt to special formats such as vertical text and Arabic text.

[0082] This embodiment, through the above-described scheme, specifically involves acquiring the original page image and the standard answer page image of the target test paper, performing hierarchical structure recognition on the original page image to obtain the question hierarchy and the hierarchical test paper skeleton; mapping the answer text in the standard answer page image to the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain the mapping relationship; detecting the answer area in the original page image to obtain the answer box area, normalizing the answer box area to obtain candidate answer boxes; correcting the candidate answer boxes to obtain the target answer box; and merging the question hierarchy, mapping relationship, and target answer box into the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain a structured grading template. Therefore, by performing hierarchical structure recognition on the original test paper image, a hierarchical test paper skeleton is generated from the page space structure. This allows the question set to be completely enumerated independently of the text order and manual labeling. Then, the test paper questions in the skeleton are used as the basis to map the answer text, and the answer area is normalized into candidate answer boxes and the coordinates are verified. Then, the question hierarchy, mapping relationship and target answer box are merged into the skeleton based on the same test paper questions. This makes the answer semantics and the answer area coordinates attached to the same primary key. This solves the problem that the existing technology lacks authoritative question skeletons that are independent of the text order and lacks a unified primary key for answer text and answer area coordinates, which leads to the problems of missing questions, duplicate questions and mismatched scoring. This improves the efficiency of cross-modal grading template generation.

[0083] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 In step S038, the method for generating a cross-modal grading template further includes steps S0381 to S0383, whereby the candidate answer box is corrected based on the verification result to obtain the target answer box. Step S0381: Based on the boundary information, crop the candidate answer boxes that cross the page boundary to obtain cropped candidate answer boxes; Step S0382: Based on the degradation information, perform region deletion on the cropped candidate answer box to obtain the retained answer box; Step S0383: Sort the reserved answer boxes according to the arrangement order information to obtain the target answer boxes; First, based on the boundary information, the candidate answer boxes that cross the page boundary are cropped according to the page boundary so that the coordinate values ​​of each box fall within the page boundary after cropping, and the cropping mark is recorded to obtain the cropped candidate answer boxes.

[0084] Based on the degradation information, the cropped candidate answer boxes are then subjected to region deletion, that is, the zero-area boxes and inverted boxes that are identified as degradation boxes are deleted, resulting in the retained answer boxes.

[0085] Finally, based on the arrangement order information, the reserved answer boxes are sorted according to their serial numbers within the question, so that the order of multiple empty answer boxes under the same question is consistent with the natural writing order, thus obtaining the target answer box.

[0086] It is important to note that in the subsequent generation of the structured grading template, the skeleton S, answer mapping M, and target answer box set B are first merged into the corresponding question nodes in the skeleton based on the test questions of the hierarchical test paper skeleton, according to the qid or sub_qid primary key.

[0087] The merging priority is three levels. The question level and order of the original test paper skeleton have the highest priority, followed by the manually corrected fields, which can cover the model fields, and the model output fields have the lowest priority.

[0088] The answer text is written to correct_answer, the visual coordinates are written to bbox, and missing fields remain editable. The fusion result adds a source tag and confidence level to each field. The sources include skeleton, optical character recognition of the answer, visual model and manual correction. The tag is saved with the template so that the source of any field can be traced afterward.

[0089] The items to be corrected are divided into three levels according to their severity. Level P0 is for structural conflicts, such as duplicate question numbers, missing parent-child relationships, and answers that cannot be uniquely mapped. These must be manually processed before they can be used for scoring. Level P1 is for coordinate conflicts, such as cropping marks and overlapping boxes. Manual confirmation is recommended. Level P2 is for low confidence, such as answer segments with confidence below the threshold. These are only provided as a hint. Items other than Level P0 do not block template generation, thus avoiding a single anomaly from rendering the entire paper unusable. The granularity of the specific grading can also be two or four levels or more. Alternatively, continuous scoring based on confidence thresholds can be used instead of discrete grading.

[0090] The editing interface allows manual modification of question numbers, question names, scores, answers, page numbers, answer boxes, question categories, and sub-question splits. It also supports moving questions across sections. Manual correction can be done on the web, desktop, or mobile devices.

[0091] Each time the data is saved, the merged results are sorted by natural question number, field normalization is performed, and integrity checks are performed to obtain a structured correction template.

[0092] Specifically, after step S04 above, which involves merging the question hierarchy, mapping relationship, and target answer box into the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain the structured grading template, the method further includes: Step S0401: Receive the save request of the structured correction template, and generate a historical version backup of the structured correction template according to the save request; Step S0402: Perform summary analysis on the structured correction template to obtain a summary of the template content; Step S0403: Extract the question scoring points of the structured grading template, and associate and store the question scoring points with the template content summary to obtain the scoring point cache. Step S0404: Upon receiving the grading request for the target test paper, perform summary analysis on the structured grading template to obtain a real-time template content summary; Step S0405: If the real-time template content summary is consistent with the template content summary, then the target test paper is scored according to the scoring point cache, and the scoring result is obtained.

[0093] Upon receiving a request to save the structured correction template, the system first generates a historical version backup of the previous version and its MD5 hash, establishing a correspondence between the version and the content digest. The version backup can use file backup, database version table, or content-addressed storage. In addition to MD5, the content digest can also use the SHA series of digest algorithms.

[0094] Subsequently, a summary analysis was performed on the structured grading template to obtain a summary of the template content, and the question scoring points in the template were extracted. The question scoring points were then associated with and stored with the template content summary to obtain a scoring point cache.

[0095] Write the fusion result P into a structured file, such as parsed_answers.json. P should contain at least the fields sub_questions, max_score, correct_answer, bbox, page, and knowledge_points.

[0096] Structured templates can also be saved as JSON, XML, relational database records, or other equivalent serializable formats. Scoring point caches can be built by template summary, question summary, or version number.

[0097] Upon receiving a grading request for the target test paper, the current structured grading template is first re-analyzed to obtain a real-time template content summary. If the real-time template content summary is consistent with the previously saved template content summary, it means that the template has not been silently changed. The grading points of the questions are then directly extracted from the grading point cache to grade the target test paper and obtain the grading result. This avoids the waiting and calculation overhead caused by repeatedly compiling grading points. If the two are inconsistent, the cache is automatically invalidated and recalculated.

[0098] The output undergoes six completeness checks: missing questions, duplicate questions, empty frames, out-of-bounds frames, out-of-range scores, and misaligned answers. Items that fail the checks are entered into the manual correction queue, without affecting the template generation and downstream scoring of other confirmed questions.

[0099] When only the standard answer is changed, pages are added, or a partial rescan is performed, the difference update is performed at the qid granularity. Only the affected qid nodes are rebuilt, the existing manual correction overlay fields on unaffected nodes are retained, and the MD5 of the updated template is recalculated so that the historical correction results can still be traced back to the corresponding version.

[0100] The grading results continue to use the same qid and bbox, so that the grading module, grade report module, grading symbol rendering module and printing module share the same coordinate anchor point. Based on this, the rendering module overlays handwritten checkmarks, crosses and correction annotations on the original paper. Based on this, the printing module outputs paper annotations, forming a coordinate system that can be generated once and reused in multiple places.

[0101] This embodiment, through the above scheme, specifically involves cropping candidate answer boxes that cross the page boundary based on the boundary information to obtain cropped candidate answer boxes; deleting regions from the cropped candidate answer boxes based on the degradation information to obtain retained answer boxes; sorting the retained answer boxes according to the arrangement order information to obtain target answer boxes; and merging the question hierarchy, mapping relationship, and target answer boxes into the corresponding questions in the layered test paper skeleton based on the test paper questions to obtain a structured grading template. Therefore, by performing hierarchical structure recognition on the original test paper image, a hierarchical test paper skeleton is generated from the page space structure. This allows the question set to be completely enumerated independently of the text order and manual labeling. Then, the test paper questions in the skeleton are used as the basis to map the answer text, and the answer area is normalized into candidate answer boxes and the coordinates are verified. Then, the question hierarchy, mapping relationship and target answer box are merged into the skeleton based on the same test paper questions. This makes the answer semantics and the answer area coordinates attached to the same primary key. This solves the problem that the existing technology lacks authoritative question skeletons that are independent of the text order and lacks a unified primary key for answer text and answer area coordinates, which leads to the problems of missing questions, duplicate questions and mismatched scoring. This improves the efficiency of cross-modal grading template generation.

[0102] For example, to help understand the implementation flow of the cross-modal correction template generation method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 3 , Figure 3 A simplified flowchart of a method for generating cross-modal correction templates is provided, specifically: Figure 3 In this context, T represents the set of original exam paper pages, A represents the set of standard answer pages, S represents the authoritative hierarchical skeleton obtained from visual recognition of the original exam paper, M represents the answer mapping, B represents the set of candidate answer boxes, P represents the fused structured grading template, qid represents the question number identifier, sub_qid represents the sub-question number identifier, and bbox represents the normalized answer box coordinates.

[0103] The parallel processing flow of the original test paper branch and the answer branch includes: the original test paper branch sequentially completes image input, orientation correction and size unification, original test paper optical character recognition, page layout visual inspection and skeleton extraction, and completes spatial aggregation of question numbers and answer areas during skeleton extraction, outputting a hierarchical skeleton S. The answer branch sequentially completes answer optical character recognition, text normalization and question number level parsing, outputting an answer mapping M. At the same time, the page-by-page visual answer box extraction produces an answer box set B under the constraint of skeleton S, and after normalization and four-level verification, it is incorporated into the fusion stage.

[0104] The process of merging the coordinate verification results of the two branches and entering the cross-modal fusion process includes: using qid or sub_qid as the primary key during fusion, and following the merging priority of the original volume skeleton hierarchy and order as the highest priority, followed by manual correction, and then the model output as the lowest priority. During the fusion process, a source tag and confidence level are added to each field. For fields that cannot be uniquely aligned, have coordinate conflicts, or have incomplete structures, three levels of items to be corrected are generated: P0, P1, and P2. After the fusion results are corrected by the manual correction interface, natural question number sorting, field normalization, and six completeness checks are performed, and a version backup and MD5 summary are generated. Finally, a structured correction template P is output.

[0105] The downstream reuse process of the output P includes: the scoring module compiles scoring points based on the question-level fields and uses the MD5 of P as the scoring point cache key; the grade report module statistically analyzes the error distribution according to knowledge points; the correction symbol rendering module and the printing module use the same set of qid and bbox to overlay proofreading symbols and correction annotations on the original paper; the skeleton cache uses the batch identifier as the key and the scoring point cache uses the template MD5 as the key, and they are respectively bypassed on two branches.

[0106] As a result, the question set is uniquely determined by the original paper's visual structure, and the answer text can only be attached to existing questions. The answer semantics and the coordinates of the answer area are unified under the same question number primary key. Missing questions, duplicate questions, mismatched scoring tasks, and offset marking marks are eliminated. The template also has the ability to be auditable, reproducible, and incrementally maintained.

[0107] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the cross-modal correction template generation method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0108] This application also provides a cross-modal correction template generation device, please refer to... Figure 4 The cross-modal correction template generation device includes: The recognition module 10 is used to acquire the original page image of the target test paper and the standard answer page image, and to perform hierarchical structure recognition on the original page image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton. The mapping module 20 is used to map the answer text in the standard answer page image to the layered test paper skeleton according to the test paper questions of the layered test paper skeleton, so as to obtain the mapping relationship; Detection module 30 is used to detect the answering region of the original page image, obtain the answering box region, and normalize the answering box region to obtain candidate answering boxes; The correction module 40 is used to correct the candidate answer box to obtain the target answer box, and according to the test questions of the hierarchical test paper skeleton, merge the question hierarchy relationship, mapping relationship and target answer box into the hierarchical test paper skeleton to obtain a structured grading template.

[0109] The cross-modal grading template generation device provided in this application, employing the cross-modal grading template generation method described in the above embodiments, can solve the technical problems in the prior art, such as the lack of authoritative question skeletons independent of text order and the lack of a unified primary key between the answer text and the coordinates of the answer area, leading to missed questions, duplicate questions, and mismatched scoring. Compared with the prior art, the beneficial effects of the cross-modal grading template generation device provided in this application are the same as those of the cross-modal grading template generation method provided in the above embodiments, and other technical features in the cross-modal grading template generation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0110] This application provides a cross-modal correction template generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the cross-modal correction template generation method in Embodiment 1 above.

[0111] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a cross-modal correction template generation device suitable for implementing embodiments of this application. The cross-modal correction template generation device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The cross-modal correction template generation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0112] like Figure 5As shown, the cross-modal correction template generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the cross-modal correction template generation device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the cross-modal correction template generation device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show cross-modal correction template generation devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0113] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0114] The cross-modal grading template generation device provided in this application, employing the cross-modal grading template generation method described in the above embodiments, can solve the technical problems in the prior art, such as the lack of authoritative question skeletons independent of text order and the lack of a unified primary key between the answer text and the coordinates of the answer area, leading to missed questions, duplicate questions, and mismatched scoring. Compared with the prior art, the beneficial effects of the cross-modal grading template generation device provided in this application are the same as those of the cross-modal grading template generation method provided in the above embodiments, and other technical features in this cross-modal grading template generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0115] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0117] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the cross-modal correction template generation method in the above embodiments.

[0118] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0119] The aforementioned computer-readable storage medium may be included in the cross-modal correction template generation device; or it may exist independently and not be assembled into the cross-modal correction template generation device.

[0120] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the cross-modal grading template generation device, the cross-modal grading template generation device: acquires the original page image and the standard answer page image of the target test paper; performs hierarchical structure recognition on the original page image to obtain the question hierarchy and the hierarchical test paper skeleton; maps the answer text in the standard answer page image to the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain the mapping relationship; performs answer area detection on the original page image to obtain the answer box area; performs normalization processing on the answer box area to obtain the candidate answer box; corrects the candidate answer box to obtain the target answer box; and merges the question hierarchy, mapping relationship, and target answer box into the hierarchical test paper skeleton based on the test paper questions in the hierarchical test paper skeleton to obtain a structured grading template.

[0121] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0123] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0124] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned cross-modal grading template generation method. This addresses the technical problems in the prior art, such as the lack of authoritative question skeletons independent of text order and the absence of a unified primary key between the answer text and the coordinates of the answer area, leading to missed questions, duplicate questions, and mismatched scoring. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the cross-modal grading template generation method provided in the above embodiments, and will not be elaborated upon here.

[0125] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the cross-modal correction template generation method described above.

[0126] The computer program product provided in this application can solve the technical problems of existing technologies, such as the lack of authoritative question skeletons independent of text order and the lack of a unified primary key between the answer text and the coordinates of the answer area, leading to missed questions, duplicate questions, and mismatched scoring. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the cross-modal grading template generation method provided in the above embodiments, and will not be repeated here.

[0127] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating cross-modal correction templates, characterized in that, The method for generating cross-modal correction templates includes: Obtain the original page image and the standard answer page image of the target test paper, and perform hierarchical structure recognition on the original page image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton; The step of performing hierarchical structure recognition on the original test paper page image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton includes: The original page image is corrected to obtain the corrected original page image; Optical character recognition is performed on the corrected original test paper page image to obtain the original test paper text block. Based on the original test paper text block and the corrected original test paper page image, hierarchical structure recognition is performed to obtain the initial test paper skeleton and the question hierarchy relationship between questions and sub-questions in the initial test paper skeleton. The sub-question is the lower-level question node of the question and the sub-question is the question node with the lowest level. The initial test paper skeleton was subjected to a structural consistency check, and the check results were obtained. Based on the inspection results, the nodes that passed the inspection in the initial test paper skeleton are retained, and the nodes that failed the inspection are registered as test paper items to be corrected. The test paper items to be corrected are manually corrected to obtain the corrected nodes. The nodes that pass the inspection and the corrected nodes are summarized to obtain the candidate test paper skeleton. The candidate test paper skeleton is associated with the question hierarchy to obtain a hierarchical test paper skeleton; Based on the test questions in the hierarchical test paper skeleton, the answer text in the standard answer page image is mapped to the hierarchical test paper skeleton to obtain the mapping relationship; The step of mapping the answer text in the standard answer page image to the hierarchical test paper skeleton based on the test paper questions of the hierarchical test paper skeleton, and obtaining the mapping relationship, includes: Optical character recognition is performed on the standard answer page image to obtain the answer text block, and the answer text in the answer text block is normalized to obtain the normalized answer text; The question number is parsed according to the question number hierarchy parsing rule set to obtain the question number corresponding to the answer text; Based on the test paper questions in the layered test paper skeleton, the question number is matched with the questions in the layered test paper skeleton, while keeping the question structure and question order of the layered test paper skeleton unchanged, to obtain the mapping relationship between the answer text and the question. The answer texts that do not correspond to questions in the hierarchical test paper skeleton are registered as answer items to be corrected, and no new questions are added to the hierarchical test paper skeleton based on the answer items to be corrected. The original page image is subjected to answer region detection to obtain answer box regions. The answer box regions are then normalized to obtain candidate answer boxes. The candidate answer boxes are modified to obtain the target answer boxes. Based on the test questions in the hierarchical test paper skeleton, the question hierarchy, mapping relationship and target answer boxes are merged into the hierarchical test paper skeleton to obtain a structured grading template.

2. The cross-modal correction template generation method as described in claim 1, characterized in that, The steps of detecting the answer region in the original page image to obtain the answer box region, and normalizing the answer box region to obtain candidate answer boxes include: Based on the layered test paper skeleton, the answer area of ​​the original test paper page image is detected to obtain the initial answer box area and the questions corresponding to the initial answer box area; The layered test paper skeleton is subjected to question anchor frame detection. If a question anchor frame is detected, the initial answer frame area is replaced by the question anchor frame to obtain a replacement answer frame area. If no question anchor frame is detected, the initial answer frame area is used as the replacement answer frame area. Conflict resolution is performed on the replacement answer box area to obtain the answer box area; Based on the page width and page height corresponding to the original page image, the answer box area is normalized to obtain candidate answer boxes.

3. The cross-modal correction template generation method as described in claim 1, characterized in that, The step of modifying the candidate answer box to obtain the target answer box includes: The coordinates of the candidate answer box are compared with the page boundaries of the original page image to obtain boundary information; The degradation information of the candidate answer box is obtained by comparing the coordinates of the upper left corner and the lower right corner of the candidate answer box. Based on the question attribution information of the original page image, the candidate answer boxes are sorted to obtain the sorting order information; Based on the out-of-bounds information, degradation information, and arrangement order information, a verification result is generated, and the candidate answer box is corrected according to the verification result to obtain the target answer box.

4. The cross-modal correction template generation method as described in claim 3, characterized in that, The step of correcting the candidate answer box based on the verification result to obtain the target answer box includes: Based on the boundary information, the candidate answer boxes that cross the page boundary are cropped to obtain cropped candidate answer boxes; Based on the degradation information, the cropped candidate answer boxes are deleted to obtain the retained answer boxes; The reserved answer boxes are sorted according to the arrangement order information to obtain the target answer boxes.

5. The cross-modal correction template generation method as described in claim 1, characterized in that, After the step of merging the hierarchical relationship, mapping relationship, and target answer box of the test questions according to the hierarchical test paper skeleton into the hierarchical test paper skeleton to obtain the structured grading template, the method further includes: Receive a save request for the structured correction template, and generate a backup of the historical version of the structured correction template based on the save request; The structured correction template is analyzed to obtain a summary of the template content. Extract the question scoring points from the structured grading template, and associate and store the question scoring points with the template content summary to obtain a scoring point cache; Upon receiving the grading request for the target test paper, a summary analysis is performed on the structured grading template to obtain a real-time template content summary; If the real-time template content summary is consistent with the template content summary, then the target test paper is scored based on the question scoring points extracted from the scoring point cache to obtain the scoring result.

6. A cross-modal correction template generation device, characterized in that, The cross-modal correction template generation device includes: The recognition module is used to acquire the original page image of the target test paper and the standard answer page image, and to perform hierarchical structure recognition on the original page image to obtain the hierarchical relationship of the questions and the hierarchical test paper skeleton. The recognition module is also used to perform image correction on the original roll page image to obtain a corrected original roll page image. Optical character recognition is performed on the corrected original test paper page image to obtain the original test paper text block. Based on the original test paper text block and the corrected original test paper page image, hierarchical structure recognition is performed to obtain the initial test paper skeleton and the question hierarchy relationship between questions and sub-questions in the initial test paper skeleton. The sub-question is the lower-level question node of the question and the sub-question is the question node with the lowest level. The initial test paper skeleton was subjected to a structural consistency check, and the check results were obtained. Based on the inspection results, the nodes that passed the inspection in the initial test paper skeleton are retained, and the nodes that failed the inspection are registered as test paper items to be corrected. The test paper items to be corrected are manually corrected to obtain the corrected nodes. The nodes that pass the inspection and the corrected nodes are summarized to obtain the candidate test paper skeleton. The candidate test paper skeleton is associated with the question hierarchy to obtain a hierarchical test paper skeleton; The mapping module is used to map the answer text in the standard answer page image to the hierarchical test paper skeleton based on the test paper questions of the hierarchical test paper skeleton, so as to obtain the mapping relationship; The mapping module is further configured to perform optical character recognition on the standard answer page image to obtain answer text blocks, and to normalize the answer text in the answer text blocks to obtain normalized answer text. The question number is parsed according to the question number hierarchy parsing rule set to obtain the question number corresponding to the answer text; Based on the test paper questions in the layered test paper skeleton, the question number is matched with the questions in the layered test paper skeleton, while keeping the question structure and question order of the layered test paper skeleton unchanged, to obtain the mapping relationship between the answer text and the question. The answer texts that do not correspond to questions in the hierarchical test paper skeleton are registered as answer items to be corrected, and no new questions are added to the hierarchical test paper skeleton based on the answer items to be corrected. The detection module is used to detect the answering region of the original page image, obtain the answering box region, and perform normalization processing on the answering box region to obtain candidate answering boxes; The correction module is used to correct the candidate answer boxes to obtain the target answer boxes, and to merge the question hierarchy, mapping relationship and target answer boxes into the hierarchical test paper skeleton according to the test paper questions of the hierarchical test paper skeleton to obtain a structured grading template.

7. A cross-modal correction template generation device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the cross-modal correction template generation method as described in any one of claims 1 to 5.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the cross-modal correction template generation method as described in any one of claims 1 to 5.

Images