Document annotation method and device, computer equipment and storage medium

Abstract

The invention discloses a document annotation method. The method comprises the following steps: acquiring a document image to be annotated; performing first optical character recognition processing on the basis of the to-be-annotated document image to obtain a first character recognition result and corresponding character coordinates; performing second optical character recognition processing on the basis of the to-be-annotated document image to obtain a second character recognition result; according to the character coordinates, mapping the second character recognition result to the first character recognition result to obtain a target document; and annotating the to-be-annotated document image based on the target document to obtain an annotated document image. The character coordinate information obtained by the first optical character recognition processing and the high-accuracy recognition result provided by the second optical character recognition processing are combined, and the two types of recognition results are effectively fused on the basis of a character coordinate mapping strategy, so that a target document which contains character coordinates and has high text accuracy is generated; and the annotation accuracy is effectively improved according to the target document.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of text processing, and in particular to a document annotation method and device, computer equipment and a storage medium. BACKGROUND

[0002] In the education and teaching scene, especially for the evaluation and correction process of students' written homework (such as papers, exercises, etc.), it is a key link of teaching feedback and personalized tutoring. Traditionally, this process relies on teachers to manually review paper homework, identify grammar, spelling, syntax, and factual errors, and provide feedback by adding symbols, underlines, annotations, and other forms on paper. Although such manual correction methods have certain advantages in accuracy and pertinence, the operation process is time-consuming and laborious, the feedback cycle is long, and the results may be inconsistent due to differences in teachers' subjective judgments, especially in large class teaching or batch correction scenarios, there is a serious scalability bottleneck.

[0003] Currently, to improve the efficiency of written homework correction, existing technologies attempt to use optical character recognition (OCR) technology to convert paper homework scan images into editable text, thereby achieving partial automation. However, standard OCR engines, although having basic character recognition capabilities, often have limited recognition accuracy due to image quality, writing style, or complexity of layout, which in turn affects the effectiveness of subsequent analysis and processing. And in terms of feedback presentation, existing technologies mostly convert recognized text into pure text format output feedback, which is disconnected from students' original handwritten or printed documents, lacks layout consistency and visual relevance, and is difficult to intuitively present error locations and modification suggestions. The document annotation accuracy of existing technologies is low.

[0004] Therefore, how to provide a document annotation method that can effectively improve the accuracy of document annotation has become a problem to be solved. SUMMARY

[0005] Therefore, it is necessary to provide a document annotation method, device, computer equipment and storage medium to solve the problem of low accuracy of traditional document annotation methods.

[0006] A document annotation method comprises: obtaining a document image to be annotated; performing first optical character recognition processing based on the document image to be annotated to obtain a first character recognition result and corresponding character coordinates; performing second optical character recognition processing based on the document image to be annotated to obtain a second character recognition result, the recognition accuracy of the second optical character recognition processing being higher than the recognition accuracy of the first optical character recognition processing; mapping the second character recognition result to the first character recognition result according to the character coordinates, to obtain a target document; annotating the document image to be annotated based on the target document, to obtain an annotated document image.

[0007] Optionally, the annotating the document image to be annotated based on the target document, to obtain an annotated document image, comprises: providing the target document to a preset text correction model, so that the text correction model corrects the target document, to obtain a corrected document; comparing the corrected document with the target document, to obtain an editing operation type of the target document; based on the editing operation type and the character coordinates, rendering an annotation mark on the document image to be annotated, to obtain the annotated document image.

[0008] Optionally, the editing operation type comprises a replacement type and a deletion type, the replacement type corresponds to a character to be replaced, and the deletion type corresponds to a character to be deleted. The rendering an annotation mark on the document image to be annotated based on the editing operation type and the character coordinates comprises: when the editing operation type is the replacement type or the deletion type, based on the character coordinates, calculating a continuous bounding box of the character to be replaced or the character to be deleted; drawing a line through the character in the continuous bounding box to indicate the replacement or deletion operation of the character.

[0009] Optionally, the editing operation type comprises an insertion type, the insertion type corresponds to an insertion point and text to be inserted. The rendering an annotation mark on the document image to be annotated based on the editing operation type and the character coordinates comprises: determining a text length of the text to be inserted; comparing the text length with a preset length threshold, to obtain a comparison result; based on the insertion point and the comparison result, determining an insertion position of the text to be inserted; rendering the text to be inserted at the insertion position.

[0010] Optionally, the determining an insertion position of the text to be inserted based on the insertion point and the comparison result comprises: based on the insertion point, determining an upper available space and a lower available space of a corresponding text line in the document image to be annotated; if the comparison result is that the text length is less than the preset length threshold, the upper available space or the lower available space is determined as the insertion position, and a text rendering color of the upper available space is different from a text rendering color of the lower available space; if the comparison result is that the text length is greater than or equal to the preset length threshold, the insertion position is a preset side note region, and a connection line is drawn between the preset side note region and the insertion point; Optionally, the method further comprises: when the side note region is used to carry a plurality of inserted texts, the vertical positions of the inserted texts are adjusted so that the plurality of inserted texts are visually not overlapped with each other.

[0011] Optionally, the method further comprises: in the process of inserting the text to be inserted into the upper available space or the lower available space, if a same text line includes a plurality of insertion points, the rendering positions of the inserted texts of the insertion points of the same text line are adjusted so that the plurality of inserted texts are visually not overlapped with each other.

[0012] Optionally, the method further comprises: based on a plurality of the editing operation types, a statistical analysis is performed on the user's text errors of the document image to be annotated, to obtain an analysis result; based on the statistical analysis result, a guidance suggestion is generated to guide the user.

[0013] A document annotation apparatus comprises: a first acquisition module configured to acquire a document image to be annotated; a first recognition module configured to perform a first optical character recognition processing based on the document image to be annotated, to obtain a first character recognition result and corresponding character coordinates; a second recognition module configured to perform a second optical character recognition processing based on the document image to be annotated, to obtain a second character recognition result, and an identification accuracy of the second optical character recognition processing is higher than an identification accuracy of the first optical character recognition processing; a first mapping module configured to map the second character recognition result to the first character recognition result at the character coordinates, to obtain a target document; a first annotation module configured to perform an annotation processing on the document image to be annotated based on the target document, to obtain an annotated document image.

[0014] A computer device comprises a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, and the processor implements the above document annotation method when executing the computer readable instructions.

[0015] A readable storage medium has computer readable instructions stored thereon, and the computer readable instructions are executed by a processor to implement the document annotation method.

[0016] The above document annotation method, device, computer device and storage medium, obtain a document image to be annotated; based on the document image to be annotated, first optical character recognition processing is performed to obtain a first character recognition result and corresponding character coordinates; based on the document image to be annotated, second optical character recognition processing is performed to obtain a second character recognition result, and the recognition accuracy of the second optical character recognition processing is higher than that of the first optical character recognition processing; according to the character coordinates, the second character recognition result is mapped to the first character recognition result to obtain a target document; based on the target document, the document image to be annotated is annotated to obtain an annotated document image. By combining the character coordinate information obtained by the first optical character recognition processing and the high-accuracy recognition result provided by the second optical character recognition processing, the effective fusion of the two types of recognition results is realized based on the character coordinate mapping strategy, so as to generate a target document containing accurate spatial position information and having high text accuracy. On this basis, further annotation processing is performed on the document image to be annotated according to the target document, accurate visual annotation of the document image to be annotated is realized, and the performance of the correction feedback in intuitiveness, accuracy and intelligence is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flowchart of a document annotation method according to an embodiment of the present application; Figure 2 is a structural diagram of an annotated document image according to an embodiment of the present application; Figure 3 is a structural diagram of a document annotation device according to an embodiment of the present application; Figure 4 is a schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of the present application.

[0020] In an embodiment, as shown in Figure 1 A document annotation method is provided, comprising the following steps: 101. Obtain a document image to be annotated.

[0021] In the embodiments of the present application, the above-mentioned document annotation method can be applied to a text processing platform, which can be constructed by a server or a server cluster. The server or server cluster can be any electronic device with functions of text processing, text analysis, data transmission, data storage, etc. The text processing platform can be deployed with a document annotation system, through which the above-mentioned document annotation method can be implemented. The document annotation system can specifically implement any one of the document annotation methods in the embodiments of the present application. For specific reference to the limitations of the document annotation method, details are not described here to avoid repetition and redundancy.

[0022] The above-mentioned document image to be annotated can be collected by a user terminal device and transmitted to a text processing platform through a network. Image collection can be completed by shooting or scanning using devices such as smartphones, tablet computers, digital cameras, scanners, etc. The generated document image file can be in common static picture formats such as JPEG, PNG, BMP, etc., or other image formats convenient for subsequent processing.

[0023] Application scenarios include students' compositions, exercise books, test papers, reading reports, and other written work, as well as thesis drafts, contract texts, technical reports, and other documents that need to be text-checked and annotated. After image collection is completed, the document image can be sent to the text processing platform through a client application, a web interface, or other data uploading methods for subsequent recognition and annotation processing.

[0024] In a possible embodiment, the text processing platform is connected with a campus homework management platform, an online education platform, or an office automation platform to realize automatic acquisition of document images. For example, after a student completes online homework submission, the generated homework image or PDF file can be directly pushed to the annotation processing flow, omitting the manual uploading step and improving overall processing efficiency.

[0025] 102. Perform first optical character recognition processing based on the document image to be annotated to obtain a first character recognition result and corresponding character coordinates.

[0026] In the embodiment of the present application, the first optical character recognition processing can be implemented by using a standard OCR engine, which can extract the text content in the document image while ensuring the integrity of the character spatial position information. During the recognition process, the document image is divided according to the layout structure, column and row analysis, and the recognition result and spatial coordinates are generated character by character. The character coordinates can include the starting position, ending position and boundary box range of the character in the document image, which are used for subsequent accurate positioning of the annotation position. The spatial mapping accuracy of the character is prioritized, so that even in the case of image quality or text style affecting the recognition accuracy, stable positioning reference data can be provided for annotation rendering.

[0027] 103. performing a second optical character recognition processing based on the document image to be annotated, to obtain a second character recognition result.

[0028] The recognition accuracy of the second optical character recognition processing is higher than that of the first optical character recognition processing. The second optical character recognition processing can be implemented by using an artificial intelligence OCR model based on deep learning, which extracts features and predicts sequences of the document image through an end-to-end convolutional neural network, recurrent neural network or Transformer structure, to generate a recognition result. This recognition method can provide higher recognition accuracy than the first optical character recognition processing in the case of multiple fonts, multiple languages, complex layout and low-quality image environment. The second character recognition result focuses more on the restoration accuracy of the text content, but usually does not output or guarantee the accurate spatial coordinate information of the character.

[0029] 104. mapping the second character recognition result to the first character recognition result according to the character coordinates, to obtain a target document.

[0030] In the embodiment of the present application, the character coordinate data output by the first optical character recognition processing can be used as a mapping reference, and the high-accuracy character content output by the second optical character recognition processing can be matched one by one with the coordinate data. During the matching process, the two types of recognition results can be aligned according to the order information of the characters, the layout structure and the size of the character boundary box, to ensure the accurate correspondence between the text content and the spatial position information.

[0031] In the case of inconsistent number of characters or character segmentation differences, a character alignment algorithm such as the shortest edit distance comparison, dynamic time warping (DTW) or sliding window-based matching method can be introduced to insert, delete or replace misplaced or missing characters, to ensure the integrity of the mapping relationship. After the mapping is completed, the generated target document has high-accuracy text content and accurate character spatial coordinates, which can be visualized on the original document image and provide a stable data basis for subsequent intelligent correction, difference identification and other processing steps.

[0032] Specifically, the LLM alignment algorithm can be used to map the second character recognition result to the first character recognition result according to the character coordinates, thereby generating the target document with accurate layout.

[0033] It can be understood that the first optical character recognition processing is implemented by using a traditional OCR engine, which generally performs recognition on characters in a document image based on algorithms such as layout analysis, character segmentation, and rule matching, and simultaneously provides character content and accurate spatial coordinate information of the character in the image when outputting. This type of recognition method has an advantage in character positioning accuracy, and can generate stable and reproducible coordinate data for each character, facilitating subsequent superposition of annotations on the original document image. However, the recognition accuracy of traditional OCR is relatively limited when dealing with low-resolution images, complex fonts, or mixed-language content, and may result in character replacement, omission, or misjudgment.

[0034] The second optical character recognition processing uses an artificial intelligence OCR model based on deep learning, which uses structures such as convolutional neural networks, recurrent neural networks, or Transformers for end-to-end feature extraction and sequence prediction of document images, and has stronger character form generalization ability and context semantic inference ability. In complex layouts, low-quality scans, or mixed-language environments with multiple fonts, it can provide higher text recognition accuracy than the first optical character recognition processing. However, since the deep learning OCR model is usually optimized for pure text output, the spatial coordinate information of the characters in its output results is missing or not accurate enough to be directly used for original image annotation positioning.

[0035] Using the first optical character recognition processing alone can obtain accurate character coordinate information, but due to insufficient recognition accuracy, it may result in deviations between the annotation content and the actual text, reducing the reliability of the feedback.

[0036] Using the second optical character recognition processing alone can obtain high-precision text content and strong restoration ability, but due to the lack of accurate spatial position information, it cannot accurately render the annotation position on the original document image.

[0037] By executing the first optical character recognition processing and the second optical character recognition processing (which can be understood as double OCR processing) on the same document image in sequence, spatial coordinate data and high-precision text results are obtained, and the two types of results are fused based on a character coordinate mapping strategy in subsequent steps, a target document with both high accuracy and accurate spatial positioning ability can be generated. The target document not only ensures that the annotation position is consistent with the original text, but also improves the correctness and visualization effect of the correction content, thereby solving the technical problem that existing OCR solutions cannot balance accuracy and positioning accuracy.

[0038] 105. The target document is used to annotate the document image to be annotated to obtain an annotated document image.

[0039] In the embodiment of the present application, the target document can be input into a text correction model to generate a corrected text (which can also be replaced by a standard answer); then based on the comparison between the corrected text (or the standard answer) and the target document, the editing operation type (such as insertion, deletion, or replacement) and the corresponding text range are determined; then using the character coordinates bound in the target document, each editing operation is mapped to the corresponding position of the original document image; finally, according to the operation type, the annotation mark is rendered, and the annotated document image is output. Different types of editing can be presented through differentiated visualized marking methods, so that the user can intuitively understand the modified content and its position. For example, the deletion type annotation can draw a through line on the related characters, the insertion type annotation can add the corresponding text above or below the insertion point, and the replacement type annotation can display a line through the original characters and present the replacement text in the adjacent area.

[0040] The specific annotation method can be set according to actual needs, which is not limited here, for example, the annotation content can be displayed in the form of a floating label, an annotation bubble, or color highlighting to adapt to different terminal devices and user habits; an annotation index area can also be generated at the edge of the document image, and the corresponding modification position is indicated through a connecting line, so as to keep the layout clean in the content-intensive page.

[0041] For another example, the annotation processing can be combined with an interactive function, so that the user can selectively expand or fold the annotation information when viewing the document, or click on the annotation mark to view detailed instructions and modification suggestions, thereby realizing the combination of the correction result and learning guidance.

[0042] Alternatively, the annotation method can also be set according to the document type of the document image to be annotated, for example, academic papers can preferentially use side notes and symbolic marks, test papers or exercise books can preferentially use through lines, circles, and inserted words, and contracts or legal documents can use highlighting and side note indexes, thereby meeting the annotation needs of different use scenarios.

[0043] The annotation rendering can be generated in the form of image layering, which can be completed on a local device or rendered by a server and then sent to the user, facilitating the synchronous use of multiple terminals.

[0044] The above annotated document image can be further illustrated by a structure diagram of an annotated document image as shown in Figure 2 As shown in Figure 2 differentiated annotation methods are used for different editing operation types: When the editing operation type is replacement or deletion, a line through the characters is drawn in the continuous bounding box of the corresponding characters; When the editing operation type is insertion, the to-be-inserted text is rendered above or below the insertion point, and a connection line can be drawn between the insertion point and the comment content; In a text-intensive area, the inserted text can be rendered in a preset marginal note area, and a connection line is used to indicate the corresponding insertion point.

[0045] In this way, the annotated document image can display the modification content and its position in an intuitive and clear manner while maintaining the original layout, thereby improving the accuracy and readability of the annotation and being suitable for various scenarios such as educational correction and document proofreading.

[0046] In the embodiment of the present application, a document image to be annotated is obtained; first optical character recognition processing is performed based on the document image to be annotated to obtain a first character recognition result and corresponding character coordinates; second optical character recognition processing is performed based on the document image to be annotated to obtain a second character recognition result, and the recognition accuracy of the second optical character recognition processing is higher than that of the first optical character recognition processing; the second character recognition result is mapped to the first character recognition result according to the character coordinates to obtain a target document; and annotation processing is performed on the document image to be annotated based on the target document to obtain an annotated document image. By combining the character coordinate information obtained by the first optical character recognition processing and the high-accuracy recognition result provided by the second optical character recognition processing, and based on the character coordinate mapping strategy, the two types of recognition results are effectively fused to generate a target document that contains accurate spatial position information and has high text accuracy. On this basis, further annotation processing is performed on the document image to be annotated based on the target document to realize precise visual annotation of the document image to be annotated, thereby effectively improving the performance of the correction feedback in terms of intuitiveness, accuracy and intelligence.

[0047] It can be understood that in the specific embodiments of the present application, data related to document images and the like are involved, and when the embodiments in the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data, the construction and use of the text correction model need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0048] Optionally, in the step of performing annotation processing on the document image to be annotated based on the target document to obtain an annotated document image, the target document can also be provided to a preset text correction model to enable the text correction model to perform correction processing on the target document to obtain a corrected document; comparison processing is performed based on the corrected document and the target document to obtain an editing operation type of the target document; and based on the editing operation type and the character coordinates, an annotation mark is rendered on the document image to be annotated to obtain an annotated document image.

[0049] In the embodiment of the present application, the target document can be input into a preset text correction model to generate a corrected document. The text correction model can be a rule-based text processing engine, or a natural language processing model based on machine learning or deep learning, and specifically can be an LLM model (i.e., a large language model). By analyzing and correcting the content of the target document, a corrected document with higher text accuracy can be obtained.

[0050] On this basis, the corrected document and the target document are compared for differences to determine various types of editing operations and the corresponding text ranges. The difference comparison method is not limited, and can be implemented using a character-level, word-level or sentence-level matching algorithm.

[0051] In combination with the character coordinate information of the target document, each editing operation is mapped to the corresponding position of the original document image, and the annotation mark is rendered according to the editing type to obtain an annotated document image. This method not only preserves the layout of the original document, but also intuitively presents the text correction result in a visual manner, and is suitable for different types of documents and display terminals, especially for educational scenarios, such as correcting homework and thesis documents.

[0052] Optionally, the editing operation type includes a replacement type and a deletion type, the replacement type corresponds to a character to be replaced, and the deletion type corresponds to a character to be deleted. In the step of rendering the annotation mark on the document image to be annotated based on the editing operation type and the character coordinates, when the editing operation type is the replacement type or the deletion type, the continuous bounding box of the character to be replaced or the character to be deleted can be calculated based on the character coordinates; a line that penetrates the character is drawn in the continuous bounding box to indicate the replacement or deletion of the character.

[0053] In the embodiment of the present application, when the editing operation type is the replacement type or the deletion type, the continuous bounding box of the character to be replaced or the character to be deleted is first determined based on the character coordinate information. The determination of the continuous bounding box can be realized by aggregating the minimum bounding rectangles of adjacent characters to ensure that the boundary range is accurately matched with the actual text area.

[0054] Subsequently, a line that penetrates the character body is drawn in the continuous bounding box to intuitively indicate that the corresponding text has been deleted or will be replaced. For example, the penetrating line can be red or other high-visibility colors that form a clear contrast with the main text to improve the recognition. In the case of a replacement operation, the replacement text can also be displayed above, below or adjacent to the bounding box, so that the user can directly compare the original character and the new character.

[0055] In another alternative way, the line style can be adjusted according to the document type or user preference, such as a dashed line, a wavy line or a semi-transparent mask, to adapt to the visual effect requirements of different reading scenarios.

[0056] In this way, the replacement or deletion operation can be presented in an intuitive and unified visual form without changing the overall layout of the original document, thereby improving the understandability and acceptability of the annotation.

[0057] Optionally, the editing operation type includes an insertion type, the insertion type corresponds to an insertion point and to-be-inserted text, and in the step of rendering the annotation mark on the document image to be annotated based on the editing operation type and the character coordinates, the text length of the to-be-inserted text can be determined; the text length is compared with a preset length threshold to obtain a comparison result; the insertion position of the to-be-inserted text is determined based on the insertion point and the comparison result; and the to-be-inserted text is rendered at the insertion position.

[0058] In the embodiment of the present application, the preset length threshold can be dynamically set according to the document type, the layout density, or the resolution of the display terminal.

[0059] When the editing operation type is the insertion type, the text length of the to-be-inserted text can be determined first, and compared with the preset length threshold to obtain a comparison result; then the insertion position of the to-be-inserted text is determined based on the insertion point and the comparison result; and finally, the to-be-inserted text is rendered at the insertion position to intuitively present the inserted annotation in the original document image.

[0060] During rendering, the font size of the inserted text can be scaled, the line spacing can be adjusted, or the local content can be slightly displaced to adapt to the limited space layout. For long text that cannot be completely presented in the original line or adjacent blank area, the inserted content can be automatically split into multiple segments, and a connection mark can be added between each segment to indicate that they belong to the same insertion operation.

[0061] In this way, the annotation presentation position can be flexibly selected according to the length of the inserted text and the document layout, while ensuring the readability of the annotation and maximizing the preservation of the layout structure and visual neatness of the original document.

[0062] Optionally, in the step of determining the insertion position of the to-be-inserted text based on the insertion point and the comparison result, the upper available space and the lower available space of the corresponding text line in the document image to be annotated can also be determined based on the insertion point; if the comparison result is that the text length is less than the preset length threshold, the upper available space or the lower available space is determined as the insertion position, and the text rendering color of the upper available space is different from that of the lower available space; if the comparison result is that the text length is greater than or equal to the preset length threshold, the insertion position is a preset side annotation area, and a connection line is drawn between the preset side annotation area and the insertion point.

[0063] In the embodiments of the present application, when the insertion position of the text to be inserted is determined based on the insertion point and the comparison result, the corresponding text line in the document image to be annotated can be first located based on the insertion point, and the available space above and the available space below the text line are detected.

[0064] When the comparison result is that the length of the text is less than the preset length threshold, the available space above or the available space below can be selected as the insertion position, and different rendering colors can be used to distinguish them, for example, the text inserted above uses a first color, and the text inserted below uses a second color, so that the user can quickly identify the insertion direction.

[0065] When the comparison result is that the length of the text is greater than or equal to the preset length threshold, the insertion position can be set as the preset marginal note area of the document image, and a connection line can be drawn between the inserted text in the marginal note area and the insertion point to indicate the corresponding position of the inserted content in the original text.

[0066] This way can take into account the readability of the content and the aesthetic appearance of the page layout when inserting annotations, and through the intuitive identification of colors and connection lines, the user's understanding efficiency of the annotation information can be improved.

[0067] Optionally, when the marginal note area is used to carry multiple inserted texts, the vertical positions of the inserted texts can be adjusted to make the multiple inserted texts visually non-overlapping.

[0068] In the embodiments of the present application, when the marginal note area is used to carry multiple inserted texts, the vertical positions of the inserted texts can be automatically adjusted to avoid visual overlap. Specifically, the number of inserted texts already existing in the marginal note area and the vertical height occupied by the inserted texts can be first counted, and then the non-overlapping vertical positions can be assigned to the text to be inserted according to the height requirement of the text to be inserted.

[0069] In one possible embodiment, the corresponding inserted texts can be arranged in order from top to bottom according to the line order of the insertion point in the original document, to ensure that the reading order is consistent with the logic of the original text; in another implementation, the nearest and empty vertical position in the marginal note area can be dynamically calculated according to the vertical position of the insertion point on the page, so as to maximize the use of the marginal note space.

[0070] In this way, even when there are a large number of annotations in the marginal note area, the clarity and legibility of the annotation content and the neatness and beauty of the page layout can be maintained.

[0071] Optionally, during the process of inserting the text to be inserted into the available space above or the available space below, if there are multiple insertion points in the same text line, the rendering positions of the inserted texts of the multiple insertion points in the same text line can be adjusted to make the multiple inserted texts visually non-overlapping.

[0072] In the embodiments of the present application, when inserting the to-be-inserted text into the upper available space or the lower available space, if there are multiple insertion points in the same text line, the to-be-inserted text corresponding to each insertion point in the text line can be accurately adjusted in the rendering position to avoid the visual overlap of multiple to-be-inserted texts.

[0073] Specifically, the character coordinates of all insertion points in the text line can be obtained first, and the insertion points can be sorted according to the horizontal direction order of the coordinates. Subsequently, starting from the leftmost insertion point, the horizontal range occupied by the corresponding to-be-inserted text after rendering is calculated in sequence, and compared with the preset safety interval of the adjacent to-be-inserted text. If it is detected that overlap will occur, the following adjustments can be made: Horizontal microshift: slightly shifting the subsequent to-be-inserted text in the horizontal direction until the horizontal interval between the previous to-be-inserted text reaches the preset value; Text scaling: in the case of insufficient space, the font size of the to-be-inserted text can be reduced in proportion to display it completely in the available space without affecting readability; Line folding: for long to-be-inserted texts that still cannot avoid overlap, they can be displayed in multiple lines while keeping the association with the insertion points.

[0074] For the insertion in the up-down direction (i.e., there are both upper to-be-inserted texts and lower to-be-inserted texts), the system can independently calculate the layout of the upper available space and the lower available space respectively, to ensure that multiple to-be-inserted texts in the same direction do not interfere with each other, and to avoid the vertical occlusion or overlap of texts in the up-down direction.

[0075] In a possible embodiment, each insertion point in the same text line can also be allocated an independent display area, the width of which can be adaptively adjusted according to the remaining space, and the font of the to-be-inserted text can be scaled or its display style can be changed (such as omitting part of the characters and adding an ellipsis mark) when necessary. This way can maintain good page neatness and annotation readability in the dense annotation scenario.

[0076] Through the above optimization strategy, even if multiple annotations are inserted in the same text line, all annotations can be ensured not to overlap visually, significantly improving the efficiency and comfort of users reading annotation information.

[0077] Optionally, the method can further statistically analyze the user's text errors of the to-be-annotated document image based on the multiple editing operation types, to obtain an analysis result; and generate a guidance suggestion for guiding the user based on the statistical analysis result.

[0078] In the embodiments of the present application, when multiple editing operation types (including insertion, deletion, replacement, etc.) are obtained and the annotation is completed, the text errors of the user in the to-be-annotated document image can be further statistically analyzed based on these editing operation types to generate analysis results that can be used for personalized learning guidance and teaching management.

[0079] Specifically, the difference information of the correction document and the target document can be first stored in a structured manner, the category (such as spelling error, syntax error, word error, punctuation error, format error, etc.), position and context information of each error are extracted, and the occurrence frequency and distribution in the current document and historical documents are counted.

[0080] In one possible embodiment, the historical annotation records of a single user can be aggregated and analyzed to identify high-frequency error patterns and error-prone knowledge points. Based on these error patterns, a pre-set large language model (LLM) can be called to generate personalized feedback comments that are encouraging, targeted and actionable, such as pointing out weaknesses in word choice or syntax and providing suggestions for improvement. At the same time, the LLM can also automatically generate customized adaptive learning exercises according to the error patterns to help users train in their weak areas.

[0081] In another possible embodiment, the error data of a user in multiple documents can be summarized in chronological order to generate a visual personal performance statistical analysis view. This view can show the error frequency curve, the proportion distribution of different error types and the progress trend, so that the user can intuitively understand the changes and improvement space in the learning process.

[0082] For teachers or teaching managers, the error data of students in a class or a specific group can be aggregated on the server side to generate an overall performance analysis report, including the average error rate of the class, the ranking of common error types, the overall progress trend, etc. Based on these common errors, the LLM can also generate or recommend new class exercise tasks to ensure that the teaching content is accurately matched with the actual weaknesses of the students.

[0083] In the above manner, the present application not only provides highly personalized learning feedback and exercise suggestions for individual users, but also provides global and visual class learning portraits for teachers and managers, thereby realizing a closed-loop support from single-point annotation to overall teaching strategy optimization.

[0084] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0085] In an embodiment, a document annotation apparatus is provided, which corresponds to the document annotation method in the above embodiment. As shown in the figure, the document annotation apparatus comprises a first acquisition module 301, a first identification module 302, a second identification module 303, a first mapping module 304, and a first annotation module 305. The functions of the modules are described in detail as follows. Figure 3 The first acquisition module 301 is configured to acquire a document image to be annotated. The first identification module 302 is configured to perform a first optical character recognition on the document image to be annotated to obtain a first character recognition result and corresponding character coordinates. The second identification module 303 is configured to perform a second optical character recognition on the document image to be annotated to obtain a second character recognition result, wherein the recognition accuracy of the second optical character recognition is higher than that of the first optical character recognition. The first mapping module 304 is configured to map the second character recognition result to the first character recognition result according to the character coordinates to obtain a target document. The first annotation module 305 is configured to perform annotation processing on the document image to be annotated based on the target document to obtain an annotated document image.

[0086] Optionally, the first annotation module 305 is further configured to: provide the target document to a preset text correction model to enable the text correction model to perform correction processing on the target document to obtain a corrected document; perform comparison processing on the corrected document and the target document to obtain an editing operation type of the target document; render an annotation mark on the document image to be annotated based on the editing operation type and the character coordinates to obtain the annotated document image.

[0087] Optionally, the editing operation type comprises a replacement type and a deletion type, the replacement type corresponds to a character to be replaced, and the deletion type corresponds to a character to be deleted. The first annotation module 305 is further configured to: when the editing operation type is the replacement type or the deletion type, calculate a continuous bounding box of the character to be replaced or the character to be deleted based on the character coordinates; draw a line through the character in the continuous bounding box to indicate the replacement or deletion operation of the character.

[0088] Optionally, the editing operation type comprises an insertion type, the insertion type corresponds to an insertion point and text to be inserted, and the first annotation module 305 is further configured to: ​determining a text length of the to-be-inserted text; performing comparison processing based on the text length and a preset length threshold to obtain a comparison result; determining an insertion position of the to-be-inserted text based on the insertion point and the comparison result; rendering the to-be-inserted text at the insertion position.

[0089] Optionally, the first annotation module 305 is further configured to: determine an upper available space and a lower available space of a corresponding text line in the to-be-annotated document image based on the insertion point; if the comparison result is that the text length is less than the preset length threshold, determining the upper available space or the lower available space as the insertion position, the text rendering color of the upper available space being different from the text rendering color of the lower available space; if the comparison result is that the text length is greater than or equal to the preset length threshold, the insertion position is a preset side annotation region, and a connection line is drawn between the insertion point and the preset side annotation region; Optionally, the document annotation apparatus further comprises: a vertical adjustment module configured to, when the side annotation region is used to carry a plurality of inserted texts, adjust vertical positions of the plurality of inserted texts so that the plurality of inserted texts are visually non-overlapped.

[0090] Optionally, the document annotation apparatus further comprises: a horizontal adjustment module configured to, in the process of inserting the to-be-inserted text into the upper available space or the lower available space, if a same text line includes a plurality of insertion points, adjusting rendering positions of inserted texts of the plurality of insertion points in the same text line so that the plurality of inserted texts are visually non-overlapped.

[0091] Optionally, the document annotation apparatus further comprises: an analysis module configured to, based on a plurality of the editing operation types, statistically analyze text errors of a user of the to-be-annotated document image to obtain an analysis result; a generation module configured to, based on the statistical analysis result, generate a guidance suggestion for guiding the user.

[0092] The specific limitations of the document annotation apparatus can refer to the limitations of the document annotation method described above, and will not be repeated here. Each module in the above document annotation apparatus can be implemented by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0093] In one embodiment, a computer device is provided, which can be a terminal device, and its internal structure diagram can be as shown in Figure 4 The computer device includes a processor, a memory and a network interface connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer readable instructions. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer readable instructions are executed by the processor to implement a document annotation method. The readable storage medium provided in the embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0094] In the embodiments of the present application, a computer device is provided, which includes a memory, a processor and computer readable instructions stored in the memory and executable on the processor. When the processor executes the computer readable instructions, the steps of the above document annotation method are implemented.

[0095] In the embodiments of the present application, a readable storage medium is provided, which stores computer readable instructions. When the processor executes the computer readable instructions, the steps of the above document annotation method are implemented.

[0096] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through computer readable instructions, and the computer readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer readable instructions are executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0097] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0098] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A document annotation method, characterized in that, include: Get the image of the document to be annotated; Based on the document image to be annotated, a first optical character recognition process is performed to obtain the first character recognition result and the corresponding character coordinates; A second optical character recognition process is performed on the document image to be annotated to obtain a second character recognition result. The recognition accuracy of the second optical character recognition process is higher than that of the first optical character recognition process. Based on the character coordinates, the second character recognition result is mapped to the first character recognition result to obtain the target document; Annotate the document image to be annotated based on the target document to obtain an annotated document image.

2. The document annotation method as described in claim 1, characterized in that, The step of annotating the document image to be annotated based on the target document to obtain an annotated document image includes: The target document is provided to a preset text correction model so that the text correction model can perform correction processing on the target document to obtain a corrected document; Based on the comparison between the corrected document and the target document, the editing operation type of the target document is obtained; Based on the editing operation type and the character coordinates, annotation marks are rendered on the document image to be annotated to obtain the annotated document image.

3. The document annotation method as described in claim 2, characterized in that, The editing operation types include replacement and deletion types. The replacement type corresponds to a character to be replaced, and the deletion type corresponds to a character to be deleted. The step of rendering annotation marks on the document image to be annotated based on the editing operation type and the character coordinates includes: When the editing operation type is a replacement type or a deletion type, the continuous bounding box of the character to be replaced or the character to be deleted is calculated based on the character coordinates; Draw a line through the character within the continuous bounding box to indicate the replacement or deletion operation of the character.

4. The document annotation method as described in claim 2, characterized in that, The editing operation type includes an insertion type, which corresponds to an insertion point and text to be inserted. The step of rendering annotation marks on the document image to be annotated based on the editing operation type and the character coordinates includes: Determine the length of the text to be inserted; A comparison process is performed based on the text length and a preset length threshold to obtain the comparison result; Based on the insertion point and the comparison result, the insertion position of the text to be inserted is determined; The text to be inserted is rendered at the insertion position.

5. The document annotation method as described in claim 4, characterized in that, Determining the insertion position of the text to be inserted based on the insertion point and the comparison result includes: Based on the insertion point, determine the available space above and below the corresponding text line in the document image to be annotated; If the comparison result is that the text length is less than the preset length threshold, then the upper available space or the lower available space is determined as the insertion position, and the text rendering color of the upper available space is different from the text rendering color of the lower available space; If the comparison result indicates that the text length is greater than or equal to the preset length threshold, then the insertion position is a preset side note area, and a connecting line is drawn between the preset side note area and the insertion point. The method further includes: When the marginal note area is used to hold multiple inserted texts, the vertical position of each inserted text is adjusted so that the multiple inserted texts do not visually overlap.

6. The document annotation method as described in claim 5, characterized in that, The method further includes: During the process of inserting the text to be inserted into the available space above or below, If a single line of text contains multiple insertion points, the rendering positions of the inserted text at each insertion point in the same line of text are adjusted so that the multiple inserted texts do not visually overlap.

7. The document annotation method as described in claim 2, characterized in that, The method further includes: Based on multiple editing operation types, statistical analysis is performed on the text errors of the user in the document image to be annotated, and the analysis results are obtained. Based on the statistical analysis results, guidance suggestions are generated to guide the user.

8. A document annotation device, characterized in that, include: The first acquisition module is used to acquire the document image to be annotated; The first recognition module is used to perform first optical character recognition processing based on the document image to be annotated, and obtain the first character recognition result and the corresponding character coordinates; The second recognition module is used to perform second optical character recognition processing based on the document image to be annotated, and obtain a second character recognition result. The recognition accuracy of the second optical character recognition processing is higher than that of the first optical character recognition processing. The first mapping module is used to map the second character recognition result to the first character recognition result according to the character coordinates to obtain the target document; The first batch of annotation modules is used to annotate the document image to be annotated based on the target document, so as to obtain an annotated document image.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the document annotation method as described in any one of claims 1 to 7.

10. A readable storage medium having computer-readable instructions stored thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the document annotation method as described in any one of claims 1 to 7.

Images