Document chapter title identification method and device, equipment, medium and program product
By fusing visual and textual features of document page images and historical chapter titles using a multimodal large model, the problem of strong OCR dependence in existing technologies is solved, achieving highly accurate and adaptive recognition of document chapter titles.
Patent Information
- Application Number
- CN202511810814.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-02-27
AI Technical Summary
Existing technologies rely on OCR accuracy in document chapter title recognition, which has weak adaptability and results in insufficient recognition accuracy.
A multimodal large model is adopted. By traversing the images of each page of the document, the current page image and the list of historical chapter titles are input into the model. Visual and text features are fused, and progressive processing and global consistency verification are used to improve recognition accuracy.
It improves the accuracy of document chapter title recognition, ensures that the output follows the reading order, adapts to different document formats, and enhances the comprehensiveness and accuracy of recognition.
Smart Images
Figure CN121582953A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, device, medium, and program product for identifying document chapter titles. Background Technology
[0002] Currently, automatic document chapter title recognition primarily relies on text analysis methods based on Natural Language Processing (NLP) and image-to-text recognition methods combining Computer Vision (CV) and Optical Character Recognition (OCR) technology. In existing solutions, the typical process involves first transcribing the document image into text using OCR, and then employing rule-based regular expression matching or deep learning-based text classification models to identify and extract chapter titles. This method has the advantage of low implementation threshold and wide applicability, but it heavily depends on the accuracy of OCR and has limited adaptability to document layout. Summary of the Invention
[0003] The purpose of this application is to provide a document chapter title recognition method, apparatus, device, medium, and program product to improve the accuracy of document chapter title recognition.
[0004] In a first aspect, embodiments of this application provide a method for recognizing document chapter titles, including: Iterate through each page image of the document to be identified, input the current page image and the list of historical chapter titles into a pre-established multimodal large model to obtain the current chapter title list; wherein, if the current page image is the first page image of the document to be identified, then the list of historical chapter titles is the initial chapter title list, otherwise the list of historical chapter titles is the current chapter title list obtained last time; After traversing the document to be identified, the final list of current chapter titles is determined as the target chapter title list of the document to be identified.
[0005] In the above implementation process, by traversing each page image of the document to be identified, the current page image and the list of historical chapter titles are input into a pre-established multimodal large model to obtain the current chapter title list. If the current page image is the first page image of the document to be identified, the list of historical chapter titles is the initial list of chapter titles; otherwise, the list of historical chapter titles is the previously obtained current chapter title list. After traversing the document to be identified, the final current chapter title list is determined as the target chapter title list of the document to be identified. A progressive processing method can be adopted, continuously inputting the current page image and the list of historical chapter titles into the multimodal large model, and continuously using the multimodal large model to extract and fuse the features of the chapter titles of the document to be identified from the current page image and the list of historical chapter titles, finally obtaining a list of chapter titles that includes all page chapter titles of the document to be identified, thereby improving the accuracy of document chapter title recognition.
[0006] Furthermore, the step of traversing each page image of the document to be identified includes: The images of each page of the document to be identified are traversed in ascending order of page number.
[0007] In the above implementation process, by traversing each page image of the document to be recognized in ascending order of page number, it is possible not only to ensure that all page images of the document to be recognized are completely traversed, but also to ensure that the final output list of target chapter titles follows the order in which the user reads the document.
[0008] Furthermore, the current page image is the first page image of the document to be identified; Before inputting the current page image and the list of historical chapter titles into a pre-built multimodal large model to obtain the current chapter title list, the following steps are also included: Load a pre-defined chapter title tree; wherein the chapter title tree contains a virtual root node; Convert the chapter title tree into the initial chapter title list; The initial list of chapter titles is determined as the historical list of chapter titles.
[0009] In the above implementation process, when the current page image is the first page image of the document to be recognized, a pre-set chapter title tree containing a virtual root node is loaded, the chapter title tree is converted into an initial chapter title list, and the initial chapter title list is determined as the historical chapter title list. This allows chapter titles from the first page image to be added under the virtual root node in the chapter title tree, ensuring that the hierarchical order of the various chapter titles in the first page image is not disordered, thereby further improving the accuracy of document chapter title recognition.
[0010] Furthermore, the current page image is the first page image of the document to be identified; Before inputting the current page image and the list of historical chapter titles into a pre-built multimodal large model to obtain the current chapter title list, the following steps are also included: Determine the knowledge domain of the document to be identified; Obtain the initial list of chapter titles that match the knowledge domain; The initial list of chapter titles is determined as the historical list of chapter titles.
[0011] In the above implementation process, by determining the knowledge domain of the document to be identified when the current page image is the first page image of the document to be identified, obtaining an initial list of chapter titles that matches the knowledge domain, and defining the initial list of chapter titles as the historical list of chapter titles, the system can adaptively select the initial list of chapter titles that matches the knowledge domain of the document to be identified. This helps the multimodal large model to quickly identify the chapter titles in the current page image, thereby further improving the accuracy of document chapter title recognition.
[0012] Furthermore, the multimodal large model includes a visual encoder, a text encoder, and a feature fusion layer; The step of inputting the current page image and the list of historical chapter titles into a pre-established multimodal large model to obtain the current chapter title list includes: Input the current page image and the list of historical chapter titles into the multimodal large model; The visual encoder extracts visual features of each current chapter title in the current page image; wherein, the visual features include one or more of the font style and layout position of the chapter title. The text encoder extracts text features from each historical chapter title in the historical chapter title list; wherein, the text features include the text content of the chapter title and any one or more pieces of information in the hierarchical structure. The feature fusion layer fuses the visual features of the current chapter title text and the text features of the historical chapter titles to obtain the current chapter title list.
[0013] In the above implementation process, by selecting a multimodal large model including a visual encoder, a text encoder, and a feature fusion layer, the visual encoder extracts the visual features of each current chapter title in the current page image, the text encoder extracts the text features of each historical chapter title in the historical chapter title list, and the feature fusion layer fuses the visual features of each current chapter title text and the text features of each historical chapter title to obtain the current chapter title list. This multimodal large model can comprehensively and accurately extract and fuse the features of the chapter titles of the document to be identified from the current page image and the historical chapter title list, thereby further improving the accuracy of document chapter title recognition.
[0014] Furthermore, the step of inputting the current page image and the list of historical chapter titles into a pre-established multimodal large model includes: The target data is obtained by filling the current page image and the list of historical chapter titles into a predefined prompt word template; The target data is input into the multimodal large model.
[0015] In the above implementation process, by pre-defining a prompt word template, the current page image and a list of historical chapter titles are filled into the prompt word template to obtain target data. The target data is then input into the multimodal large model, which can use the standard prompt word template to guide the multimodal large model to accurately identify the chapter titles in the current page image, thereby further improving the accuracy of document chapter title recognition.
[0016] Furthermore, before determining the final obtained list of current chapter titles as the target list of chapter titles for the document to be identified, the method further includes: The final list of current chapter titles is determined to pass a global consistency check; wherein the global consistency check includes one or more of the following: chapter title number continuity check and hierarchical continuity check.
[0017] In the above implementation process, by determining the final current chapter title list as the target chapter title list of the document to be identified after the final current chapter title list passes the global consistency check, it is possible to ensure that the final output target chapter title list of the document to be identified is accurate, which is conducive to further improving the accuracy of document chapter title recognition.
[0018] Furthermore, after determining the final obtained list of current chapter titles as the target list of chapter titles for the document to be identified, the method further includes: The target chapter title list is converted into a target format and the converted target chapter title list is output; wherein, the target format includes any one or more of JSON format, XML format and outline view format compatible with document processing software.
[0019] In the above implementation process, after determining the final current chapter title list as the target chapter title list of the document to be identified, the target chapter title list is further converted into any one or more target formats, such as JSON format, XML format, and outline view format compatible with document processing software, and the converted target chapter title list is output. This can convert the target chapter title list of the document to be identified into the target format desired by the user, making it convenient for the user to directly use the target chapter title list later.
[0020] Secondly, embodiments of this application provide a document chapter title recognition device, comprising: The chapter title recognition module is used to traverse each page image of the document to be recognized, input the current page image and the historical chapter title list into a pre-established multimodal large model to obtain the current chapter title list; wherein, if the current page image is the first page image of the document to be recognized, then the historical chapter title list is the initial chapter title list, otherwise the historical chapter title list is the previously obtained current chapter title list; The chapter title output module is used to determine the final current chapter title list as the target chapter title list of the document to be identified after traversing the document to be identified.
[0021] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; when the processor executes the computer program, it implements the method described above.
[0022] Fourthly, embodiments of this application provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the method described above.
[0023] Fifthly, embodiments of this application provide a computer program product, the computer program product including instructions, which, when executed by a computer, cause the computer to perform the method described above. Attached Figure Description
[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 A flowchart illustrating a document chapter title recognition method provided in the first embodiment of this application; Figure 2 A schematic diagram of the structure of a document chapter title recognition device provided in the second embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of this application. Detailed Implementation
[0026] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0027] It should be noted that in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. Furthermore, the step numbers in the text are only for the convenience of explaining the embodiments of this application and are not intended to limit the order in which the steps are performed.
[0028] In related technologies, the automatic recognition of document chapter titles mainly relies on text analysis methods based on Natural Language Processing (NLP) and image-to-text recognition methods combining Computer Vision (CV) and Optical Character Recognition (OCR) technologies. In existing solutions, the typical process is to first transcribe the document image into text using OCR, and then use rule-based regular expression matching or deep learning-based text classification models to identify and extract chapter titles. This method has the advantage of low implementation threshold and wide applicability, but it heavily depends on the accuracy of OCR and has weak adaptability to document layout.
[0029] To address this, this application proposes a document chapter title recognition method. By traversing each page image of the document to be recognized, the current page image and a list of historical chapter titles are input into a pre-established multimodal large model to obtain the current chapter title list. If the current page image is the first page image of the document to be recognized, the historical chapter title list is the initial chapter title list; otherwise, the historical chapter title list is the previously obtained current chapter title list. After traversing the document to be recognized, the final current chapter title list is determined as the target chapter title list for the document to be recognized. This method employs a progressive processing approach, continuously inputting the current page image and the historical chapter title list into the multimodal large model, and continuously using the multimodal large model to extract and fuse features related to the chapter titles of the document to be recognized from the current page image and the historical chapter title list, ultimately obtaining a chapter title list that includes all page chapter titles of the document to be recognized, thereby improving the accuracy of document chapter title recognition.
[0030] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0031] The methods provided in this application can be executed by relevant terminal devices, and the following descriptions all use user terminals as the execution subject.
[0032] It should be noted that user terminals include terminal devices such as mobile phones, tablets, and laptops that can recognize and process documents.
[0033] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a document chapter title recognition method provided in the first embodiment of this application. The document chapter title recognition method includes steps S101-S102: S101. Traverse each page image of the document to be recognized, input the current page image and the list of historical chapter titles into the pre-established multimodal large model to obtain the current chapter title list; wherein, if the current page image is the first page image of the document to be recognized, the list of historical chapter titles is the initial list of chapter titles, otherwise the list of historical chapter titles is the current chapter title list obtained last time.
[0034] As an example, users can select the document to be recognized according to their actual business needs and input the document into their user terminal.
[0035] After the user terminal obtains the document to be recognized input by the user, it determines the images of each page of the document to be recognized.
[0036] In practical applications, the user terminal can scan each page of the document to be identified to obtain images of each page of the document.
[0037] In practical applications, users can also directly input images of each page of the document to be recognized into the user terminal.
[0038] After obtaining the images of each page of the document to be recognized, the user terminal iterates through each page image. For the currently iterated page image, the historical chapter title list is obtained. If the current page image is the first page image of the document to be recognized, the historical chapter title list is the pre-set initial chapter title list; otherwise, the historical chapter title list is the previously obtained current chapter title list. The current page image and the historical chapter title list are input into a pre-established multimodal large model to obtain the current chapter title list.
[0039] It should be noted that multimodal large models are deep learning models that are jointly trained by integrating multiple types of data such as text, images, videos or audio. The core technologies include cross-modal encoder training, semantic alignment and feature fusion.
[0040] Understandably, after obtaining the images of each page of the document to be recognized, the user terminal iterates through each page image. Upon reaching the first page image, since this first page image is the first page image of the document, the historical chapter title list is determined to be the pre-set initial chapter title list. The first page image and the initial chapter title list are then input into a pre-established multimodal model to obtain the current chapter title list output by the multimodal model, which is the first chapter title list. This first chapter title list includes the initial chapter title list and the chapter titles from the first page image. The process continues until the second page image of the document is reached. Afterwards, since the second page image currently being traversed (i.e., the current page image) is not the first page image of the document to be recognized, it is determined that the historical chapter title list is the previously obtained current chapter title list, i.e., the first chapter title list. The second page image and the first chapter title list are input into the multimodal large model to obtain the current chapter title list output by the multimodal large model, which is the second chapter title list. The second chapter title list includes the chapter titles from the first chapter title list and the second page image. This operation is repeated until the document to be recognized is completely traversed, and the final current chapter title list is obtained. The final current chapter title list includes the chapter titles from all pages of the document to be recognized.
[0041] By employing a progressive processing approach, the current page image and the list of historical chapter titles are continuously input into the multimodal large model. The multimodal large model can continuously extract and fuse features of the chapter titles of the document to be identified from the current page image and the list of historical chapter titles, ultimately obtaining a list of chapter titles that includes all page chapter titles of the document to be identified.
[0042] S102. After completing the traversal of the documents to be identified, the final list of current chapter titles is determined as the target chapter title list of the documents to be identified.
[0043] As an example, after the user terminal completes the traversal of the document to be identified, it assumes that it has identified the chapter titles in all pages of the document to be identified. The final current chapter title list includes the chapter titles in all pages of the document to be identified. At this time, the final current chapter title list is determined as the target chapter title list of the document to be identified.
[0044] This application embodiment traverses each page image of the document to be identified, inputting the current page image and the historical chapter title list into a pre-established multimodal large model to obtain the current chapter title list. If the current page image is the first page image of the document to be identified, the historical chapter title list is the initial chapter title list; otherwise, the historical chapter title list is the previously obtained current chapter title list. After traversing the document to be identified, the final current chapter title list is determined as the target chapter title list of the document to be identified. This progressive processing method continuously inputs the current page image and the historical chapter title list into the multimodal large model, continuously extracting and fusing features related to the chapter titles of the document from the current page image and the historical chapter title list, ultimately obtaining a chapter title list including all page chapter titles of the document to be identified, thereby improving the accuracy of document chapter title recognition.
[0045] In an optional embodiment, traversing each page image of the document to be identified includes: traversing each page image of the document to be identified in ascending order of page number.
[0046] As an example, after obtaining the images of each page of the document to be recognized, the user terminal traverses each page image of the document in ascending order of page number.
[0047] In practical applications, user terminals can also traverse the images of each page of the document to be recognized in descending order of page number.
[0048] This application embodiment traverses each page image of the document to be identified in ascending order of page number. This not only ensures complete traversal of each page image of the document to be identified, but also ensures that the final output list of target chapter titles follows the order in which the user reads the document.
[0049] In an optional embodiment, the current page image is the first page image of the document to be identified; before inputting the current page image and the list of historical chapter titles into a pre-established multimodal large model to obtain the current chapter title list, the method further includes: loading a pre-set chapter title tree; wherein the chapter title tree contains a virtual root node; converting the chapter title tree into an initial chapter title list; and determining the initial chapter title list as the historical chapter title list.
[0050] As an example, after obtaining the images of each page of the document to be recognized, the user terminal traverses each page image of the document to be recognized. After traversing to the first page image of the document to be recognized, since the first page image traversed is the first page image of the document to be recognized, it is necessary to obtain the initial chapter title list at this time so as to determine the initial chapter title list as the historical chapter title list.
[0051] To improve the accuracy of recognizing the image on the first page of a document, a chapter title tree containing a common virtual root node can be pre-defined. For example, a pre-defined chapter title tree is shown below: { “level”: 0, "title": "Document root", "page": 0, "children": [] }
[0052] The specific process by which the user terminal obtains the initial chapter title list is as follows: First, a pre-set chapter title tree containing a virtual root node is loaded; then, the chapter title tree is converted into the initial chapter title list; finally, the initial chapter title list is determined as the historical chapter title list.
[0053] After that, the first page image and the initial list of chapter titles can be input into the pre-established multimodal large model to obtain the current list of chapter titles output by the multimodal large model, which is the first list of chapter titles. The first list of chapter titles includes the initial list of chapter titles and the chapter titles in the first page image.
[0054] This application embodiment, when the current page image is the first page image of the document to be identified, loads a pre-set chapter title tree containing a virtual root node, converts the chapter title tree into an initial chapter title list, and determines the initial chapter title list as the historical chapter title list. This allows chapter titles from the first page image to be added under the virtual root node in the chapter title tree, ensuring that the hierarchical order of the various chapter titles in the first page image is not disordered, thereby further improving the accuracy of document chapter title recognition.
[0055] In an optional embodiment, the current page image is the first page image of the document to be identified; before inputting the current page image and the list of historical chapter titles into a pre-established multimodal large model to obtain the current chapter title list, the method further includes: determining the knowledge domain of the document to be identified; obtaining an initial list of chapter titles that matches the knowledge domain; and determining the initial list of chapter titles as the list of historical chapter titles.
[0056] As an example, after obtaining the images of each page of the document to be recognized, the user terminal traverses each page image of the document to be recognized. After traversing to the first page image of the document to be recognized, since the first page image traversed is the first page image of the document to be recognized, it is necessary to obtain the initial chapter title list at this time so as to determine the initial chapter title list as the historical chapter title list.
[0057] Considering that in practical application scenarios, the chapter titles in documents from different knowledge domains are usually different, if the document to be identified is an academic paper in a certain knowledge domain, the chapter titles in the document to be identified are relatively fixed. Multiple knowledge domains can be set in advance, and a general initial list of chapter titles can be set for each knowledge domain. For example, the initial list of chapter titles matching a certain knowledge domain is as follows: ["Abstract", "Introduction", "Related Work", "Methods", "Experiments", "Conclusion", "References"].
[0058] The specific process for the user terminal to obtain the initial chapter title list is as follows: First, determine the knowledge domain of the document to be identified; then, obtain the initial chapter title list that matches the knowledge domain; finally, determine the initial chapter title list as the historical chapter title list.
[0059] After that, the first page image and the initial list of chapter titles can be input into the pre-established multimodal large model to obtain the current list of chapter titles output by the multimodal large model, which is the first list of chapter titles. The first list of chapter titles includes the initial list of chapter titles and the chapter titles in the first page image.
[0060] This application embodiment determines the knowledge domain of the document to be identified when the current page image is the first page image of the document to be identified, obtains an initial list of chapter titles that matches the knowledge domain, and determines the initial list of chapter titles as the historical list of chapter titles. It can adaptively select the initial list of chapter titles that matches the knowledge domain of the document to be identified, which helps the multimodal large model to quickly identify the chapter titles in the current page image, thereby further improving the accuracy of document chapter title recognition.
[0061] In an optional embodiment, the multimodal large model includes a visual encoder, a text encoder, and a feature fusion layer. The step of inputting the current page image and the list of historical chapter titles into the pre-established multimodal large model to obtain the current chapter title list includes: inputting the current page image and the list of historical chapter titles into the multimodal large model; extracting visual features of each current chapter title in the current page image using the visual encoder; wherein the visual features include one or more of the font style and layout position of the chapter title; extracting text features of each historical chapter title in the historical chapter title list using the text encoder; wherein the text features include one or more of the text content and hierarchical structure of the chapter title; and fusing the visual features of each current chapter title text and the text features of each historical chapter title using the feature fusion layer to obtain the current chapter title list.
[0062] As an example, a visual encoder, a text encoder, and a feature fusion layer are pre-designed to build a multimodal large model including the visual encoder, text encoder, and feature fusion layer.
[0063] In practical applications, a visual encoder can be a Vision Transformer (ViT) encoder, and a text encoder can be a Transformer encoder.
[0064] After the user terminal inputs the current page image and the list of historical chapter titles into the pre-established multimodal large model, the visual encoder in the multimodal large model will acquire the current page image, extract the visual features of each current chapter title in the current page image, and input the visual features of each current chapter title into the feature fusion layer in the multimodal large model. The visual features include one or more of the following: the font style and the layout position of the chapter title.
[0065] It is understandable that the current chapter title refers to the chapter title displayed in the image on the current page.
[0066] In practical applications, the font style of chapter titles includes, but is not limited to, font type (such as SimSun or KaiTi), font size, bold, italic, underline, and font color. The layout of chapter titles includes, but is not limited to, text line indentation, bullet points corresponding to text lines, and the relative position of text lines in the current page image (such as Y-coordinate sequence).
[0067] The text encoder in the multimodal large model obtains a list of historical chapter titles, extracts the text features of each historical chapter title in the list, and inputs the text features of each historical chapter title into the feature fusion layer in the multimodal large model. The text features include the text content of the chapter titles and any one or more pieces of information in the hierarchical structure.
[0068] It is understandable that the history chapter title refers to the chapter title in the history page images, that is, the chapter title in the previously visited page images.
[0069] The feature fusion layer in the multimodal large model acquires the visual features of each current chapter title and the text features of each historical chapter title. It then fuses the visual features of each current chapter title text and the text features of each historical chapter title text to obtain a list of current chapter titles.
[0070] In practical applications, multimodal large models can achieve the following functions during processing: 1. Identify chapter titles and update the historical chapter title list to obtain the current chapter title list: The multimodal large model not only identifies new chapter titles, but also needs to determine the numbering order and hierarchical relationship between the new chapter titles and the previously identified chapter titles. The numbering and hierarchy of chapter titles can be determined by visual features (such as font size, indentation, etc.) and predefined numbering and hierarchical rules (such as numbering sequence: 1.0 -> 1.1, hierarchy sequence: 1 -> 2).
[0071] 2. Features a conflict resolution mechanism: The multimodal large model resolves potential conflicts based on the chapter titles in the current page image and the list of historical chapter titles. (1) Handling of repeated chapter titles: When the multimodal large model detects a chapter title that is highly similar to a historical chapter title, such as “Introduction” appearing again on the second page image, it will calculate the text similarity and analyze the context to determine that it is a “continuation chapter title” rather than a new chapter title, thereby avoiding duplicate addition.
[0072] (2) Hierarchical error correction: When the multimodal large model detects that the first chapter title in the current page image is visually shallower than the last chapter title in the previous page image, such as when jumping from "3.1.1 Experimental Setup" to "3. Methodology", it will recognize this as a chapter backtracking and correctly attach it to the parent title "3. Methodology".
[0073] In practical applications, multimodal large models include, but are not limited to, the InternVL model.
[0074] This application embodiment employs a multimodal large model comprising a visual encoder, a text encoder, and a feature fusion layer. The visual encoder extracts visual features from the current chapter titles in the current page image, the text encoder extracts text features from the historical chapter titles in the historical chapter title list, and the feature fusion layer fuses the visual features of the current chapter title text and the text features of the historical chapter titles to obtain the current chapter title list. This multimodal large model comprehensively and accurately extracts and fuses features of the chapter titles of the document to be identified from the current page image and the historical chapter title list, thereby further improving the accuracy of document chapter title recognition.
[0075] In an optional embodiment, the step of inputting the current page image and the list of historical chapter titles into a pre-established multimodal large model includes: filling the current page image and the list of historical chapter titles into a predefined prompt word template to obtain target data; and inputting the target data into the multimodal large model.
[0076] As an example, to precisely guide the behavior of multimodal large models, a structured prompt template can be predefined. For instance, a predefined prompt template might look like this: "You are a document structure analysis expert. Given the currently identified chapter title tree: <list of historical chapter titles>, carefully analyze the following document page images to identify all possible new chapter titles or subheadings. Pay particular attention to lines of text that are larger, bold, centered, or numbered. Your output must be an updated, well-structured JSON list of chapter titles, supplementing or correcting the original list without providing any further explanation."
[0077] After determining the current page image and the list of historical chapter titles, the user terminal calls a predefined prompt word template, fills the current page image and the list of historical chapter titles into the prompt word template to obtain the target data, and inputs the target data into the multimodal large model to obtain the current chapter title list output by the multimodal large model.
[0078] This application embodiment uses a predefined prompt word template, fills the current page image and a list of historical chapter titles into the prompt word template to obtain target data, and inputs the target data into a multimodal large model. It can use the standard prompt word template to guide the multimodal large model to accurately identify the chapter titles in the current page image, thereby further improving the accuracy of document chapter title recognition.
[0079] In an optional embodiment, before determining the final obtained list of current chapter titles as the target list of chapter titles for the document to be identified, the method further includes: determining that the final obtained list of current chapter titles passes a global consistency check; wherein, the global consistency check includes any one or more of the following: chapter title number continuity check and hierarchical continuity check.
[0080] As an example, after the user terminal completes the traversal of the document to be identified and obtains the final list of current chapter titles, it can continue to perform a global consistency check on the final list of current chapter titles. The global consistency check includes one or more of the following: chapter title number continuity check and hierarchical continuity check.
[0081] In practical applications, assuming that global consistency checks include checks on the continuity of chapter title numbers and the continuity of chapter titles, the following checks are performed on the final list of current chapter titles: First, the continuity of chapter title numbers is checked. If two adjacent chapter titles in the final list have non-continuous numbers (e.g., 3.1 and 3.3, with the number jumping directly from 3.1 to 3.3), the continuity check fails. If all chapter titles in the final list have continuous numbers, the continuity check succeeds. Second, the continuity of chapter titles is checked. If two adjacent chapter titles in the final list have non-continuous levels (e.g., 2 and 4, with the level jumping directly from 2 to 4), the continuity check fails. If all chapter titles in the final list have continuous levels, the continuity check succeeds. If both the numbering continuity check and the hierarchical continuity check are successful, the final list of current chapter titles is determined to have passed the global consistency check. If either the numbering continuity check or the hierarchical continuity check fails, the final list of current chapter titles is determined to have failed the global consistency check.
[0082] If the final list of current chapter titles passes the global consistency check, it is considered accurate and can then be used as the template chapter title list for the document to be identified.
[0083] If the final list of current chapter titles fails the global consistency check, it is considered that the final list of current chapter titles contains errors or omissions, and some chapter titles may not have been recognized. In this case, the method described in the first embodiment of this application can be re-executed, or an error can be directly reported to notify the user in a timely manner to take appropriate action, such as modifying the prompt word template.
[0084] This application embodiment determines the final current chapter title list as the target chapter title list of the document to be identified after the final current chapter title list passes the global consistency check. This ensures that the final output target chapter title list of the document to be identified is accurate and helps to further improve the accuracy of document chapter title identification.
[0085] In an optional embodiment, after determining the final current chapter title list as the target chapter title list of the document to be identified, the method further includes: converting the target chapter title list into a target format and outputting the converted target chapter title list; wherein the target format includes any one or more of JSON format, XML format, and outline view format compatible with document processing software.
[0086] As an example, in order to output a list of target chapter titles of the document to be identified as required by the user, the target format can be pre-set according to the user's requirements, wherein the target format includes any one or more of JSON format, XML format and outline view format compatible with document processing software.
[0087] After obtaining the list of target chapter titles of the document to be recognized, the user terminal converts the list of target chapter titles into the target format and outputs the converted list of target chapter titles.
[0088] This application embodiment, after determining the final current chapter title list as the target chapter title list of the document to be identified, further converts the target chapter title list into any one or more target formats among JSON format, XML format, and outline view format compatible with document processing software, and outputs the converted target chapter title list. This can convert the target chapter title list of the document to be identified into the target format desired by the user, making it convenient for the user to directly use the target chapter title list later.
[0089] Please refer to Figure 2 , Figure 2 This is a schematic diagram of a document chapter title recognition device provided in the second embodiment of this application. The second embodiment of this application provides a document chapter title recognition device 20, including: a chapter title recognition module 201, used to traverse each page image of the document to be recognized, inputting the current page image and a historical chapter title list into a pre-established multimodal large model to obtain a current chapter title list; wherein, if the current page image is the first page image of the document to be recognized, the historical chapter title list is the initial chapter title list; otherwise, the historical chapter title list is the previously obtained current chapter title list; and a chapter title output module 202, used to determine the final obtained current chapter title list as the target chapter title list of the document to be recognized after completing the traversal of the document to be recognized.
[0090] In an optional embodiment, traversing each page image of the document to be identified includes: traversing each page image of the document to be identified in ascending order of page number.
[0091] In an optional embodiment, the current page image is the first page image of the document to be identified; the chapter title recognition module 201 is further configured to perform the following operations: before inputting the current page image and the historical chapter title list into a pre-established multimodal large model to obtain the current chapter title list, loading a pre-set chapter title tree; wherein the chapter title tree contains a virtual root node; converting the chapter title tree into an initial chapter title list; and determining the initial chapter title list as the historical chapter title list.
[0092] In an optional embodiment, the current page image is the first page image of the document to be identified; the chapter title recognition module 201 is further configured to perform the following operations: before inputting the current page image and the historical chapter title list into the pre-established multimodal large model to obtain the current chapter title list, it further includes: determining the knowledge domain of the document to be identified; obtaining an initial chapter title list that matches the knowledge domain; and determining the initial chapter title list as the historical chapter title list.
[0093] In an optional embodiment, the multimodal large model includes a visual encoder, a text encoder, and a feature fusion layer. The step of inputting the current page image and the list of historical chapter titles into the pre-established multimodal large model to obtain the current chapter title list includes: inputting the current page image and the list of historical chapter titles into the multimodal large model; extracting visual features of each current chapter title in the current page image using the visual encoder; wherein the visual features include one or more of the font style and layout position of the chapter title; extracting text features of each historical chapter title in the historical chapter title list using the text encoder; wherein the text features include one or more of the text content and hierarchical structure of the chapter title; and fusing the visual features of each current chapter title text and the text features of each historical chapter title using the feature fusion layer to obtain the current chapter title list.
[0094] In an optional embodiment, the step of inputting the current page image and the list of historical chapter titles into a pre-established multimodal large model includes: filling the current page image and the list of historical chapter titles into a predefined prompt word template to obtain target data; and inputting the target data into the multimodal large model.
[0095] In an optional embodiment, the chapter title output module 202 is further configured to determine, before determining the final obtained current chapter title list as the target chapter title list of the document to be identified, that the final obtained current chapter title list passes the global consistency check; wherein, the global consistency check includes any one or more of the following: chapter title number continuity check and hierarchical continuity check.
[0096] In an optional embodiment, the chapter title output module 202 is further configured to, after determining the final current chapter title list as the target chapter title list of the document to be identified, convert the target chapter title list into a target format and output the converted target chapter title list; wherein, the target format includes any one or more of JSON format, XML format and outline view format compatible with document processing software.
[0097] The specific implementation process of the functions and roles of each module in the above-mentioned device 20 can be found in the implementation process of the corresponding steps in the method described in the first embodiment of this application, and will not be repeated here.
[0098] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of this application. The third embodiment of this application provides an electronic device 30, including a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301; when the processor 301 executes the computer program, it implements the method described in the first embodiment of this application and can achieve the same beneficial effects.
[0099] When the processor 301 reads a computer program from the memory 302 via the bus 303 and executes the computer program, it can implement any of the methods described in the first embodiment of this application.
[0100] Processor 301 can process digital signals and may include various computing architectures. For example, it may be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 301 may be a microprocessor.
[0101] The memory 302 can be used to store instructions executed by the processor 301 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 301 of this disclosure embodiment can be used to execute instructions in the memory 302 to implement the method described in the first embodiment of this application. The memory 302 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.
[0102] The fourth embodiment of this application provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to perform the method described in the first embodiment of this application, and can achieve the same beneficial effects.
[0103] The method described in the first embodiment of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the various embodiments of this application are executed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, a core network device, an OAM (Open Application Model), or other programmable devices.
[0104] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.
[0105] The fifth embodiment of this application provides a computer program product, which includes instructions that, when executed by a computer, cause the computer to perform the method described in the first embodiment of this application and achieve the same beneficial effects.
[0106] The methods described in the first embodiment of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the various embodiments of this application are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, a core network device, an OAM (Open Application Model), or other programmable devices.
[0107] Computer programs or instructions can be stored in or transferred from one computer-readable storage medium to another. For example, a computer program or instructions can be transferred from one website, computer, server, or data center to another via wired or wireless means. A computer-readable storage medium can be any usable medium that a computer can access, or a data storage device such as a server or data center that integrates one or more usable media. Usable media can be magnetic media, such as floppy disks, hard disks, and magnetic tapes; optical media, such as digital video discs; or semiconductor media, such as solid-state drives. The computer-readable storage medium can be volatile or non-volatile, or may include both types.
[0108] In summary, embodiments of this application provide a document chapter title recognition method, apparatus, device, medium, and program product. The document chapter title recognition method includes: traversing each page image of the document to be recognized, inputting the current page image and a historical chapter title list into a pre-established multimodal large model to obtain a current chapter title list; wherein, if the current page image is the first page image of the document to be recognized, the historical chapter title list is the initial chapter title list, otherwise the historical chapter title list is the previously obtained current chapter title list; after completing the traversal of the document to be recognized, the final current chapter title list is determined as the target chapter title list of the document to be recognized. This application embodiment traverses each page image of the document to be identified, inputting the current page image and the historical chapter title list into a pre-established multimodal large model to obtain the current chapter title list. If the current page image is the first page image of the document to be identified, the historical chapter title list is the initial chapter title list; otherwise, the historical chapter title list is the previously obtained current chapter title list. After traversing the document to be identified, the final current chapter title list is determined as the target chapter title list of the document to be identified. This progressive processing method continuously inputs the current page image and the historical chapter title list into the multimodal large model, continuously extracting and fusing features related to the chapter titles of the document from the current page image and the historical chapter title list, ultimately obtaining a chapter title list including all page chapter titles of the document to be identified, thereby improving the accuracy of document chapter title recognition.
[0109] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0110] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0111] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0112] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method of identifying chapter headings in a document, characterized by, The method comprises the following steps: Traverse each page image of the to-be-recognized document, input the current page image and a historical chapter title list into a pre-established multimodal large model to obtain a current chapter title list; if the current page image is the first page image of the to-be-recognized document, the historical chapter title list is an initial chapter title list; otherwise, the historical chapter title list is the last obtained current chapter title list; After the traversal of the to-be-recognized document is completed, the last obtained current chapter title list is determined as the target chapter title list of the to-be-recognized document.
2. The method of claim 1, wherein, The traversal of each page image of the to-be-recognized document comprises the following steps: Traverse each page image of the to-be-recognized document in the order from small to large page number.
3. The method of claim 1, wherein, The current page image is the first page image of the to-be-recognized document. Before the input of the current page image and the historical chapter title list into the pre-established multimodal large model to obtain the current chapter title list, the method further comprises the following steps: Load a pre-set chapter title tree; the chapter title tree contains a virtual root node; Convert the chapter title tree into the initial chapter title list; Determine the initial chapter title list as the historical chapter title list.
4. The method of claim 1, wherein, The current page image is the first page image of the to-be-recognized document. Before the input of the current page image and the historical chapter title list into the pre-established multimodal large model to obtain the current chapter title list, the method further comprises the following steps: Determine the knowledge field of the to-be-recognized document; Obtain the initial chapter title list matched with the knowledge field; Determine the initial chapter title list as the historical chapter title list.
5. The method of claim 1, wherein, The multimodal large model comprises a visual encoder, a text encoder and a feature fusion layer. The input of the current page image and the historical chapter title list into the pre-established multimodal large model to obtain the current chapter title list comprises the following steps: Input the current page image and the historical chapter title list into the multimodal large model; Extract visual features of each current chapter title in the current page image through the visual encoder; the visual features comprise any one or more of font style and layout position of the chapter title; Extract text features of each historical chapter title in the historical chapter title list through the text encoder; the text features comprise any one or more of text content and hierarchical structure of the chapter title; Fuse the visual features of each current chapter title and the text features of each historical chapter title through the feature fusion layer to obtain the current chapter title list.
6. The method of claim 1, wherein, The input of the current page image and the historical chapter title list into the pre-established multimodal large model comprises the following steps: Fill the current page image and the historical chapter title list in a pre-defined prompt word template to obtain target data; Input the target data into the multimodal large model.
7. The method of claim 1, wherein, Before the determination of the last obtained current chapter title list as the target chapter title list of the to-be-recognized document, the method further comprises the following steps: The last obtained current chapter title list is determined to pass a global consistency check; wherein the global consistency check includes any one or more of a chapter title number continuity check and a hierarchical continuity check.
8. The method according to any one of claims 1 to 7, characterized in that, After the last obtained current chapter title list is determined to be the target chapter title list of the to-be-recognized document, the method further includes: The target chapter title list is converted into a target format, and the converted target chapter title list is output; wherein the target format includes any one or more of a JSON format, an XML format, and an outline view format compatible with a document processing software.
9. A document section title recognition apparatus characterized by comprising: The method includes: a chapter title recognition module, configured to traverse each page image of a to-be-recognized document, input a current page image and a historical chapter title list into a pre-established multi-modal large model, and obtain a current chapter title list; wherein if the current page image is a first page image of the to-be-recognized document, the historical chapter title list is an initial chapter title list, otherwise the historical chapter title list is a last obtained current chapter title list; a chapter title output module, configured to determine a last obtained current chapter title list to be a target chapter title list of the to-be-recognized document after the traversal of the to-be-recognized document is completed.
10. An electronic device, comprising: The computer program is stored in the memory and configured to be executed by the processor; and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.
11. A computer readable storage medium, characterized in that, The computer readable storage medium includes a stored computer program; wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the method according to any one of claims 1 to 8 when the computer program runs.
12. A computer program product, characterised in that, The computer program product includes instructions which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 8.