A visual-based document parsing method and system

Abstract

The application discloses a kind of based on vision's document analysis method and system, applied to intelligent identification technical field, method includes: first artificial intelligence model is split into multiple fragment documents with target source document, and obtains the meta-information corresponding to fragment document;Second artificial intelligence model is parsed according to meta-information to all fragment documents to generate corresponding analysis fragment and join pagination anchor in analysis fragment, third artificial intelligence model is abnormal analysis and repair to analysis fragment;Pagination anchor is the page number identification corresponding to each fragment document;The final analysis fragment obtained is spliced to form the complete analysis document of target source document.This application is combined by intelligent splitting and parallel analysis, and is supplemented with comprehensive abnormal monitoring and recovery mechanism, effectively overcome the context length limit of existing large artificial intelligence model, guarantee the semantic and structural integrity of content unit after splitting, improve processing efficiency and system stability.

Description

Technical Field

[0001] This invention relates to intelligent information technology, specifically to a vision-based document parsing method and system. Background Technology

[0002] With the rapid development of artificial intelligence technology, large-scale AI models with visual understanding capabilities, namely multimodal large models, have demonstrated powerful document image understanding and content generation capabilities. This makes it possible to automate, end-to-end content extraction or translation directly from PDF files or image sequences composed of page images.

[0003] However, in practical applications, existing artificial intelligence models generally face a core challenge: context length limitation. That is, the amount of information a model can process at once (usually measured in tokens or pages) is limited. For documents hundreds, thousands, or even longer, it's impossible to input them all into the model for processing at once. This challenge is particularly pronounced when dealing with large-scale existing documents. Currently, there are massive amounts of PDF documents in scanned form, containing a wealth of important historical archival materials, which may have been created decades or even centuries ago. These historical scans are typically extremely long, and due to their age, paper aging, inconsistent printing quality, or distortion during the scanning process, their image quality is often low. This results in low recognition rates for traditional, non-intelligent optical character recognition (OCR) technologies, making effective information extraction difficult.

[0004] To address the context length limitation issue, existing technologies typically employ the approach of segmenting long documents into chunks. However, these methods have significant drawbacks when faced with the aforementioned complexities. A simple segmentation method involves dividing the document into fixed page numbers and then assigning each chunk to an AI model for processing. The disadvantage of this method is that its segmentation is mechanical, completely ignoring the document's inherent content structure. When a segmentation point happens to fall within a table, list, code block, or a paragraph with complete semantic meaning, it disrupts the structural integrity and semantic coherence of these content units. This causes the model to parse the chunk incorrectly due to incomplete context information, severely impacting the quality of the final generated content.

[0005] Another common approach is to first use traditional optical character recognition (OCR) technology to convert each page of the entire document into plain text, and then clean and integrate the plain text. This method is particularly unsuitable for the large number of low-quality historical scans mentioned above. Its main drawback is that traditional OCR technology has a low recognition rate, producing a large number of erroneous characters when processing these documents, leading to the loss or distortion of key information. Furthermore, this method loses the original, rich layout information of the document while extracting text, such as the hierarchical relationship of chapter titles, multi-column layout, and the correspondence between charts and text. This visual information is crucial for accurately understanding the document's structure and semantics. When traditional OCR-recognized documents contain numerous errors, and all layout information is lost, even if advanced artificial intelligence models are subsequently used, it is difficult to recover the original logical structure of the document, significantly reducing the accuracy of the parsing. Summary of the Invention

[0006] In order to at least overcome the above-mentioned shortcomings in the prior art, the purpose of this application is to provide a vision-based document parsing method and system.

[0007] In a first aspect, embodiments of this application provide a vision-based document parsing method, including:

[0008] The first artificial intelligence model acquires the target source document;

[0009] The first artificial intelligence model splits the target source document into multiple fragment documents and obtains the metadata corresponding to the fragment documents; the metadata is the descriptive information of the fragment documents.

[0010] The second AI model parses all the fragment documents based on the metadata to generate corresponding parsed fragments and adds pagination anchors to the parsed fragments; the third AI model performs abnormal parsing and repair on the parsed fragments; the second AI model and the third AI model are from the same source model; the pagination anchors are page number identifiers corresponding to each fragment document;

[0011] The resulting parsed fragments are then spliced ​​together to form the complete parsed document of the target source document.

[0012] Secondly, this application also provides a vision-based document parsing system, including:

[0013] The first artificial intelligence model is configured as follows:

[0014] Obtain the target source document;

[0015] The target source document is split into multiple fragment documents, and the metadata corresponding to the fragment documents is obtained; the metadata is the descriptive information of the fragment documents.

[0016] The second artificial intelligence model is configured to parse all the fragment documents based on the metadata to generate corresponding parsed fragments and add pagination anchors to the parsed fragments; the pagination anchors are page number identifiers corresponding to each fragment document.

[0017] The third artificial intelligence model is configured to perform anomaly analysis and repair on the parsed fragment; the second and third artificial intelligence models are from the same source.

[0018] The splicing unit is configured to splice the finally obtained parsed fragments to form a complete parsed document of the target source document.

[0019] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0020] This invention provides a vision-based document parsing method and system that combines intelligent splitting with parallel parsing, supplemented by a comprehensive anomaly monitoring and recovery mechanism. This effectively overcomes the context length limitations of existing large-scale artificial intelligence models, ensures the semantic and structural integrity of the split content units, and improves processing efficiency and system stability. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0022] Figure 1 This is a schematic diagram of the method steps in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0024] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0025] Please refer to the following: Figure 1 This is a flowchart illustrating a vision-based document parsing method provided in an embodiment of the present invention. Further, the vision-based document parsing method may specifically include the content described in steps S1-S4.

[0026] S1: The first artificial intelligence model acquires the target source document;

[0027] S2: The first artificial intelligence model splits the target source document into multiple fragment documents and obtains the metadata corresponding to the fragment documents; the metadata is the descriptive information of the fragment documents;

[0028] S3: The second artificial intelligence model parses all the fragment documents according to the metadata to generate corresponding parsed fragments and adds pagination anchors to the parsed fragments; the third artificial intelligence model performs abnormal parsing and repair on the parsed fragments; the second artificial intelligence model and the third artificial intelligence model are from the same source model; the pagination anchor is the page number identifier corresponding to each fragment document;

[0029] S4: The obtained parsed fragments are spliced ​​together to form the complete parsed document of the target source document.

[0030] In the implementation of this application, a three-stage strategy of first splitting, then parsing, and then correcting errors is adopted. The first stage requires splitting the target source document using a first artificial intelligence model. This splitting is generally based on the document's chapter structure and the principle of protecting the integrity of units that are not suitable for splitting, such as tables and images, to determine the splitting points. The target source document is preferably an image sequence composed of multiple page image files; if the artificial intelligence model supports it, it can also be a PDF. For Word documents (.doc, .docx), PPT presentations (.ppt, .pptx), etc., they can be pre-converted into image sequences or PDFs before processing.

[0031] In this embodiment, the core task of the first artificial intelligence model is to understand the overall or partial structure of the source document and formulate a reasonable splitting scheme. Therefore, the first artificial intelligence model should be selected with a longer context processing capability to more accurately understand the macro structure of the document. During splitting, it needs to obtain specific information for locating splitting points, such as page numbers, specific text on the page such as titles, or visual features on the page such as the end position of tables. A page may not contain any splitting points, or it may contain one or more splitting points. At this time, the first artificial intelligence model splits the target source document according to the obtained specific information. The form of the resulting fragment document is related to the form of the target source document and the processing capability of the model, generally in the form of an image sequence or PDF. When outputting fragment documents, the first artificial intelligence model also needs to output corresponding meta-information. Meta-information is descriptive information associated with each fragment document, which will be used to guide the second artificial intelligence model to parse the fragment documents. Meta-information may include, but is not limited to, the original start and end page numbers of the fragment document in the target source document, the chapter title to which the fragment document belongs, the chapter level, and other prompts generated by the first artificial intelligence model that help the second artificial intelligence model understand the context of the fragment document.

[0032] In this embodiment, the task of the second artificial intelligence model is to parse each segment and perform text extraction and / or translation. It needs to possess visual understanding capabilities. During parsing, it extracts the text content from the segment according to its layout and logical order, or, based on the extracted text content, translates it into another language. This embodiment does not impose many limitations. Preferably, the generated text segment is in Markdown format. Markdown format is lightweight and easy to read and write, and can be easily converted to various other formats such as Word, PDF, and HTML, making it very suitable as an intermediate and final output format. When the second artificial intelligence model outputs, it also adds corresponding pagination anchors to the parsed segment to facilitate real-time monitoring and anomaly detection of the second artificial intelligence model's output behavior. This pagination anchor is a special text mark with a predefined format. After the second artificial intelligence model completes the parsing of each page of the target source document, it inserts a corresponding pagination anchor into the generated parsed segment. The pagination anchor preferably adopts the format of HTML comments, for example...<!-- page-N --> Here, N represents the page number of the current segment, ensuring that it can be captured and parsed by the monitoring program without being displayed in the final rendered complete parsed document. To achieve this, explicit formatting requirements generally need to be added to the parsing instructions of the second artificial intelligence model. For example, during the parsing process, after extracting the content of a page, a pagination anchor must be inserted immediately on a new line, and its format must be [format missing].<!-- page-N -->N is the page number of the current page, and the parsing continues to the next page.

[0033] In this embodiment, a third artificial intelligence model, which shares the same origin as the second AI model, is also provided for anomaly detection and repair. Here, "same origin" means using the same large model, although the calling method can use different parameters. It should be understood that the third AI model is a logical functional role. Anomaly detection can be continuous or triggered based on predefined conditions, such as triggering a detection every 2000 characters or tokens generated by the second AI model. Alternatively, a structural unit-based triggering method can be used, such as triggering a detection every time a second-level heading and its corresponding content are generated by the second AI model. A hybrid strategy can also be used, for example, triggering a detection every 2000 characters or a second-level heading generated. This embodiment does not impose further limitations. When detection is triggered, the system submits the currently generated fragment from the second AI model as input to the third AI model, along with explicit detection instructions. These instructions guide the third AI model to evaluate text quality from multiple dimensions. For example, a detection instruction could be: "You are an AI output quality detection robot. Please determine whether the following text contains logical inconsistencies, factual errors, content disconnected from context, abrupt changes in language style, or meaningless repetitions. If there are no abnormalities, please simply answer 'normal.' If there are abnormalities, please answer 'abnormal' and briefly explain the reason." The third AI model makes a judgment based on this instruction, returns the detection result, and then performs repairs. Since the second and third AI models are from the same source, after the third AI model identifies a detection error, it can use different parameters than the second AI model for secondary identification. This method can ensure that the detection output results of the second and third AI models are more stable, thereby ensuring that the parsed segments of subsequent outputs remain consistent. It should be understood that the first, second, and third AI models described in this application embodiment are all large models, each requiring configuration of corresponding operating parameters and needing to be interactive and controllable through question-and-answer instructions.

[0034] In one possible implementation, the splitting of the fragment document includes:

[0035] The first artificial intelligence model identifies the chapter structure and fixed blocks of the target source document; the fixed blocks are parts that are not suitable for splitting.

[0036] The first artificial intelligence model determines the splitting point based on the chapter structure and the fixed block, and obtains the title and level of the corresponding fragment document as the descriptive information based on the chapter structure;

[0037] The first artificial intelligence model performs image segmentation on the target source document based on the splitting points to form the fragment document.

[0038] In the implementation of this application embodiment, the first artificial intelligence model needs to identify the chapter structure and fixed blocks when splitting the document. It identifies the document's semantics and chapter structure by analyzing the page visual information of the source document. For example, the identification process first attempts to locate and parse the table of contents in the target source document. If the table of contents is identified, the chapter titles, hierarchical relationships, and corresponding page numbers provided by the table of contents are used as the most prioritized and reliable basis for determining the document structure. If the target source document does not contain an identifiable table of contents or the table of contents information is incomplete, the first artificial intelligence model infers the chapter structure by comprehensively analyzing various visual and layout features of the text. For example, the model analyzes the font size, font weight (e.g., whether it is bold), line indentation, and numbering format (e.g., Chapter 1, 1.2, (a)). Through these features, the model can infer the level of different text lines in the document's hierarchical structure, such as distinguishing between first-level headings, second-level headings, and body text. After identifying the structural information such as the various levels of headings in the document through the above methods, the first artificial intelligence model can determine the optimal splitting point. For example, the starting position of each chapter or sub-chapter can be used as a split point. This method of splitting based on the semantic structure of the document can ensure that the split fragments are relatively complete and independent in terms of content, which is beneficial to the subsequent understanding and parsing of the fragments by the second artificial intelligence model.

[0039] In this embodiment, fixed blocks are visual structures within a document that are independent or possess complete semantic meaning. Once split into different segments, their content and structure will be disrupted, severely impacting subsequent parsing quality. Besides tables, lists, and code blocks, these units can also include charts and their captions, complex mathematical formulas, footnotes, or endnotes. Forcibly splitting these units in the middle may cause the second AI model to fail to correctly understand their structure, leading to information confusion, content loss, or even parsing failure. When analyzing page visual information, the first AI model identifies various fixed blocks on the page. Through visual recognition technology, the model can determine the visual boundaries of each fixed block on the page. When determining splitting points, the model ensures that the splitting points do not fall within the visual boundaries of any fixed block. A common principle in determining splitting points is to add splitting points above the chapter titles, ensuring that the splitting points do not fall within the aforementioned visual boundaries. This marking allows subsequent splitting operations to accurately start from the title line, dividing the page content into different segments.

[0040] In one possible implementation, image segmentation to form the fragment document includes:

[0041] The first artificial intelligence model retrieves a preset number of pages as a page batch, starting from the first page of the target source document; the preset number is equal to the maximum number of pages that the second artificial intelligence model reads in at one time.

[0042] When the split point is not present in the page batch, the first artificial intelligence model marks the end of the page batch with a continuation character and takes the next page of the page batch as the starting page.

[0043] When the split point exists in the page batch, and the last split point is located in the middle of the page, the first artificial intelligence model takes that page as the starting page.

[0044] When the split point exists in the page batch, and the last split point is located at the bottom of the page, the first artificial intelligence model takes the next page of that page as the starting page.

[0045] The first artificial intelligence model repeatedly selects page batches again starting from the initial page and generates new initial pages until the entire target source document is divided into page batches;

[0046] The first artificial intelligence model performs image segmentation on the corresponding page at the split point within each page batch to form the fragment document.

[0047] In this embodiment, the page batch is a preset number of pages. Adjusting this preset number allows control over the length of the split document segments, ensuring that the length of each output segment does not overflow the second AI model. When further splitting based on the split points in the page batch, the starting point for each page batch identification, i.e., the initial page, needs to be determined; each time, the page batch acquisition must begin from the initial page. This embodiment provides multiple initial page identification processes. If there are no split points in the page batch, it indicates that the page batch is a complete chapter. In this case, a continuation character needs to be marked at the end, and the next page of the page batch is taken as the starting page. The continuation character is used for continuous chapter splicing during subsequent output by the second AI model. When the final split point is located in the middle of the page, the page is split, and the lower half of the split corresponds to the starting page. When the final split point is located at the bottom of the page, the next page after the split is the starting page. Through the above logical constraints, all page batches can be finally determined and split at the split points to form document segments.

[0048] For example, suppose each page batch contains 10 pages. The processing flow is as follows: In the first processing, the starting page is page 1, so pages 1-10 are obtained to form the first page batch. These 10 pages are input into the first AI model for analysis. Suppose the model detects two split points: one in the middle of page 8 and the other at the end of page 10. The last split point in this batch is located at the end of page 10. Therefore, the starting page for the next page batch will be set to the page after page 10, i.e., page 11. In the second processing, the starting page is page 11, so pages 11-20 are obtained to form the second page batch. Suppose the model detects a split point in the middle of page 15. The last split point in this batch is located in the middle of page 15, not at the end of its page. Therefore, the starting page for the next page batch will be set to the page containing this split point, i.e., page 15. The purpose of this is to ensure that the content of page 15 can be re-analyzed as the starting point for the next batch, guaranteeing that structural units spanning multiple pages are fully included in at least one batch for analysis, thus ensuring the accuracy of the split points. In the third processing step, starting with page 15, pages 15-24 are retrieved to form the third page batch. Subsequent processing follows this pattern until all pages of the source document have been processed.

[0049] In one possible implementation, performing anomalous parsing on the parsed fragment includes:

[0050] The third artificial intelligence model obtains the pagination anchor point and performs text duplication detection on the same document segment.

[0051] When the pagination anchor point is abnormal or there is an abnormality in text repetition, the third artificial intelligence model determines that the parsed segment is abnormal.

[0052] In the implementation of this application embodiment, by continuously monitoring these pagination anchors while receiving the text stream, the system can perform various anomaly detections on the model's behavior. These anomalies typically indicate that the second artificial intelligence model may have failed to correctly understand the document structure, and its output process has "gone out of control" or fallen into an error state. Specific anomaly detection types include: page number out-of-range detection, i.e., the system knows the total number of pages in the currently parsed segment. If the page number N in the received pagination anchor exceeds this range, for example, if the segment has 10 pages, but the received page number N exceeds this range...<!-- page-11 --> Anchor points with the same page number indicate that the model is experiencing a hallucination. When this occurs more frequently than a set threshold, it is considered an anomaly. Page number duplication detection means that if the system continuously receives anchor points with the same page number, it likely means the model is stuck in a loop, repeatedly processing the same page's content. When the number of page number duplications exceeds a set threshold, it is considered an anomaly. Page number skip detection means that if the received pagination anchor point sequence is discontinuous, with jumps such as...<!-- page-3 -->Then directly<!-- page-5 --> The missing anchor point for page 4 indicates that the model may have missed parsing the entire page. An anomaly is identified when the number of page number jumps exceeds a set threshold. Page number backtracking detection occurs if the received pagination anchor point sequence shows a decreasing sequence, such as...<!-- page-4 --> Next is<!--page-3 --> This is a serious logical error, indicating that the model has completely lost the ability to correctly track the document order. An anomaly is identified when the number of page number rollbacks exceeds a set threshold. Single-page text length anomaly detection means the system can monitor the text length generated between two consecutive pagination anchors. If the text length of a page significantly exceeds a reasonable threshold, such as five times the average number of characters per page, it may indicate that the model generated a large amount of repetitive or meaningless content on that page, or that it forgot to insert the pagination anchor. The text length can be calculated in characters or, more precisely, in the number of tokens processed by the model.

[0053] In this embodiment, analysis is also required based on text repetition anomalies. These anomalies are used to monitor in real-time whether low-level loop errors exist in the output content of the second artificial intelligence model. In some cases, especially when processing visually unclear or complexly structured documents, some artificial intelligence models may fall into a state of output degradation, i.e., out of control, and begin generating repetitive and meaningless content. The anomaly detection method proposed in this embodiment can effectively capture such behavior. The first type of anomaly detection is for the detection of consecutively repeated characters. This anomaly manifests as the model continuously generating the same character in the output stream. For example, the model may continuously output a large number of newline characters, resulting in large areas of blank space in the generated document; or continuously output a large number of spaces or tabs; when parsing tables, it may also continuously output a large number of "-" characters to align the tables because it cannot correctly identify the table boundaries. To achieve detection, the system monitors the number of consecutively occurring identical characters in real-time when receiving the text stream. Once this number exceeds a preset, reasonable threshold, such as more than 200 consecutive newline characters, the system will determine that an anomaly has occurred. The second type of anomaly detection is for the detection of consecutively repeated text blocks. This type of anomaly is an extension of the aforementioned situation, manifesting as the model repeatedly generating the same phrase, sentence, or entire paragraph. For example, the model might continuously repeat "according to the following diagram, according to the following diagram, according to the following diagram," getting stuck in a logical loop from which it cannot escape. To detect this anomaly, the system can maintain a buffer of recently generated output content and set a text block size, such as 15 words. The system continuously compares the most recently generated text block in the buffer with the immediately preceding text block. If the number of consecutive repetitions exceeds a set threshold, for example, if the same text block appears 10 times consecutively, the system will determine that an anomaly has occurred.

[0054] In one possible implementation, repairing the abnormal parsed fragment includes:

[0055] The third artificial intelligence model generates an abnormal page from the abnormal parsing fragment as an abnormal page, and sends the meta-information of the abnormal page to the first artificial intelligence model.

[0056] The first artificial intelligence model performs a secondary split on the abnormal page to form multiple abnormal split pages; the secondary split is to split the abnormal page into multiple pages of fixed length;

[0057] After the third artificial intelligence model identifies each of the abnormal split pages, it replaces the abnormal part of the abnormal parsed segment with the identified and verified content.

[0058] In this embodiment, when an anomaly occurs, a secondary identification of the abnormal page is required. During this secondary identification, the first artificial intelligence model further splits the abnormal page, generating smaller pages. While the third artificial intelligence model originates from the second model, it can be configured with different parameters to specifically identify these smaller pages. The identified content can also be better integrated into the abnormal parts of the parsed fragment. Generally, the main parameters of the third and second artificial intelligence models include model temperature, model top_p, model top_k, model frequency_penalty, and model presence_penalty. The temperature value controls the randomness of the output; lower values ​​make the output more deterministic, while higher values ​​introduce more variation. If the model gets stuck in repetitive output, appropriately increasing the temperature may help break the loop. Top_p (kernel sampling) restricts the model from selecting the next word from a smaller, high-probability vocabulary set, which helps reduce the probability of the model rambling or generating content irrelevant to the context. Other parameters such as frequency_penalty and presence_penalty can be used to suppress word-level repetition, while top_k provides another way to limit the range of word selection. If all parameter adjustments fail, modifying the prompts input to the third AI model itself, such as adding more specific instructions or constraints, is a last resort to solve the problem.

[0059] For example, the default parameters for the second AI model are set to Temperature 0.5, Top-p 1.0, Frequency Penalty 0, and Presence Penalty 0. The third AI model, however, can use multiple sets of parameters to attempt repairs to address anomalies. Here is an example set of parameters: Temperature 1.0, Top-p 1.0, Frequency Penalty 0.3, and Presence Penalty 0.2. Increasing the Temperature value increases randomness, which can prevent output from spiraling out of control to some extent. Increasing the Frequency Penalty and Presence Penalty values ​​can penalize repeated outputs, often mitigating the output spiraling problem.

[0060] In one possible implementation, the generation of the parsed fragment further includes:

[0061] The second artificial intelligence model obtains the last paragraph of the first parsed segment and the first paragraph of the second parsed segment; the first parsed segment and the second parsed segment are adjacent parsed segments, and the first parsed segment is in front of the second parsed segment;

[0062] When both the last paragraph and the first paragraph are text paragraphs, the second artificial intelligence model concatenates the last paragraph and the first paragraph to form a coherent text, and analyzes the language logic, grammatical structure and document hierarchy coherence of the coherent text.

[0063] If the coherence meets the expected criteria, the second artificial intelligence model adds a continuum character to the end of the first parsed segment;

[0064] If the coherence does not meet the expected standard, the second artificial intelligence model adds a separator to the end of the first parsed segment.

[0065] In implementing this application, a further verification process for the relationship between parsed segments is provided. In practical use, the inventors have found that when restoring document content after document recognition, relying solely on hard-coded rules, such as simple string concatenation or regular expression judgments based on line ending punctuation, can lead to unnatural line breaks or paragraph overlap in the restored Markdown document when faced with complex formatting situations, such as headings spanning multiple pages. Therefore, this application requires obtaining the end and beginning paragraphs of two consecutive parsed segments and merging them for verification.

[0066] In this embodiment, the merged coherent text is validated based on language logic, grammatical structure, and document hierarchy. During this validation, the second AI model analyzes whether the language logic of the coherent text is fluent, whether the grammatical structure is complete or repetitive, and whether the document hierarchy is continuous. Only when the above coherence requirements are met is the coherent text considered compliant. At this point, a continuation character is added to the end of the first parsed segment; otherwise, a delimiter is added. In subsequent complete text generation, continuation characters connect two segments into a single paragraph, while delimiters separate two segments into two different paragraphs, thus resolving the aforementioned problem. Secondly, this application also provides a vision-based document parsing system, including:

[0067] The first artificial intelligence model is configured as follows:

[0068] Obtain the target source document;

[0069] The target source document is split into multiple fragment documents, and the metadata corresponding to the fragment documents is obtained; the metadata is the descriptive information of the fragment documents.

[0070] The second artificial intelligence model is configured to parse all the fragment documents based on the metadata to generate corresponding parsed fragments and add pagination anchors to the parsed fragments; the pagination anchors are page number identifiers corresponding to each fragment document.

[0071] The third artificial intelligence model is configured to perform anomaly analysis and repair on the parsed fragment; the second and third artificial intelligence models are from the same source.

[0072] The splicing unit is configured to splice the finally obtained parsed fragments to form a complete parsed document of the target source document.

[0073] In one possible implementation, the first artificial intelligence model is further configured as follows:

[0074] Identify the chapter structure and fixed blocks of the target source document; the fixed blocks are parts that are not suitable for splitting.

[0075] The splitting point is determined based on the chapter structure and the fixed blocks, and the title and level of the corresponding fragment document are obtained as the descriptive information based on the chapter structure.

[0076] The target source document is segmented into the fragment document based on the split points.

[0077] In one possible implementation, the first artificial intelligence model is further configured as follows:

[0078] A preset number of pages are retrieved as a page batch, starting from the first page of the target source document; the preset number is equal to the maximum number of pages that the second artificial intelligence model can read in at one time.

[0079] When the split point does not exist in the page batch, a continuation character is marked at the end of the page batch and the next page of the page batch is taken as the starting page;

[0080] When the split point exists in the page batch, and the last split point is located in the middle of the page, the page is taken as the starting page.

[0081] When the split point exists in the page batch, and the last split point is located at the bottom of the page, the next page of that page is taken as the starting page.

[0082] Repeat the process of selecting page batches again starting from the aforementioned starting page and generating new starting pages until the entire target source document is divided into page batches;

[0083] Within each page batch, the corresponding page is segmented at the split point to form the fragment document.

[0084] In one possible implementation, the third artificial intelligence model is configured as follows:

[0085] Extract pagination page numbers from the metadata and perform text duplication detection on the same document segment.

[0086] When the pagination page number is abnormal or there is an abnormality in the text repetition, it is determined that the parsed segment is abnormal.

[0087] In one possible implementation, the third artificial intelligence model is configured as follows:

[0088] The abnormal parsing fragment generates an abnormal page as the abnormal page, and the first artificial intelligence model receives multiple abnormal split pages formed by secondary splitting of the abnormal page; the secondary splitting is to split the abnormal page into multiple pages of fixed length.

[0089] After identifying each of the abnormal split pages, the abnormal part of the abnormal parsing segment is replaced with the content that has been identified and verified.

[0090] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0092] The units described as separate components may or may not be physically separate. As will be apparent to those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0093] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vision-based document parsing method, characterized in that, include: The first artificial intelligence model acquires the target source document; The first artificial intelligence model splits the target source document into multiple fragment documents and obtains the metadata corresponding to the fragment documents; the metadata is the descriptive information of the fragment documents. The second artificial intelligence model parses all the fragment documents based on the meta-information to generate corresponding parsed fragments and adds pagination anchors to the parsed fragments; the third artificial intelligence model performs abnormal parsing and repair on the parsed fragments. The second and third artificial intelligence models are from the same source; the pagination anchors are page number identifiers corresponding to each fragment document; The resulting parsed fragments are then spliced ​​together to form the complete parsed document of the target source document; Repairing the abnormal parsing fragment includes: the third artificial intelligence model generates an abnormal page from the abnormal parsing fragment as an abnormal page, and sends the meta-information of the abnormal page to the first artificial intelligence model; The first artificial intelligence model performs a secondary split on the abnormal page to form multiple abnormal split pages; the secondary split is to split the abnormal page into multiple pages of fixed length; After the third artificial intelligence model identifies each of the abnormal split pages, it replaces the abnormal part of the abnormal parsed segment with the identified and verified content.

2. The vision-based document parsing method according to claim 1, characterized in that, The segmentation of the document includes: the first artificial intelligence model identifying the chapter structure and fixed blocks of the target source document; the fixed blocks are parts that are not suitable for segmentation; The first artificial intelligence model determines the splitting point based on the chapter structure and the fixed block, and obtains the title and level of the corresponding fragment document as the descriptive information based on the chapter structure; The first artificial intelligence model performs image segmentation on the target source document based on the splitting points to form the fragment document.

3. The vision-based document parsing method according to claim 2, characterized in that, The process of segmenting an image to form the document fragment includes: the first artificial intelligence model acquiring a preset number of pages as a page batch, starting from the first page of the target source document; the preset number is equal to the maximum number of pages read in by the second artificial intelligence model at one time; When the split point is not present in the page batch, the first artificial intelligence model marks the end of the page batch with a continuation character and takes the next page of the page batch as the starting page. When the split point exists in the page batch, and the last split point is located in the middle of the page, the first artificial intelligence model takes that page as the starting page. When the split point exists in the page batch, and the last split point is located at the bottom of the page, the first artificial intelligence model takes the next page of that page as the starting page. The first artificial intelligence model repeatedly selects page batches again starting from the initial page and generates new initial pages until the entire target source document is divided into page batches; The first artificial intelligence model performs image segmentation on the corresponding page at the split point within each page batch to form the fragment document.

4. The vision-based document parsing method according to claim 1, characterized in that, The abnormal parsing of the parsed segment includes: the third artificial intelligence model obtaining the pagination anchor point and performing text duplication detection on the same segment document; When the pagination anchor point is abnormal or there is an abnormality in text repetition, the third artificial intelligence model determines that the parsed segment is abnormal.

5. The vision-based document parsing method according to claim 1, characterized in that, The generation of the parsed segment further includes: the second artificial intelligence model obtaining the end paragraph of the first parsed segment and the beginning paragraph of the second parsed segment; the first parsed segment and the second parsed segment are adjacent parsed segments, and the first parsed segment is in front of the second parsed segment; When both the last paragraph and the first paragraph are text paragraphs, the second artificial intelligence model concatenates the last paragraph and the first paragraph to form a coherent text, and analyzes the language logic, grammatical structure and document hierarchy coherence of the coherent text. If the coherence meets the expected criteria, the second artificial intelligence model adds a continuum character to the end of the first parsed segment; If the coherence does not meet the expected standard, the second artificial intelligence model adds a separator to the end of the first parsed segment.

6. A vision-based document parsing system using the method described in any one of claims 1 to 5, characterized in that, include: The first artificial intelligence model is configured as follows: Obtain the target source document; The target source document is split into multiple fragment documents, and the metadata corresponding to the fragment documents is obtained; the metadata is the descriptive information of the fragment documents. The second artificial intelligence model is configured to parse all the fragment documents based on the meta-information to generate corresponding parsed fragments and add pagination anchors to the parsed fragments; The pagination anchor point is the page number identifier corresponding to each of the fragment documents; The third artificial intelligence model is configured to perform anomaly analysis and repair on the parsed fragment; the second and third artificial intelligence models are from the same source. The splicing unit is configured to splice the finally obtained parsed fragments to form a complete parsed document of the target source document.

7. A vision-based document parsing system according to claim 6, characterized in that, The first artificial intelligence model is further configured to: identify the chapter structure and fixed blocks of the target source document; the fixed blocks are parts that are not suitable for splitting; The splitting point is determined based on the chapter structure and the fixed blocks, and the title and level of the corresponding fragment document are obtained as the descriptive information based on the chapter structure. The target source document is segmented into the fragment document based on the split points.

8. A vision-based document parsing system according to claim 7, characterized in that, The first artificial intelligence model is further configured to: retrieve a preset number of pages as a page batch, starting from the first page of the target source document; the preset number is equal to the maximum number of pages read in one go by the second artificial intelligence model; When the split point does not exist in the page batch, a continuation character is marked at the end of the page batch and the next page of the page batch is taken as the starting page; When the split point exists in the page batch, and the last split point is located in the middle of the page, the page is taken as the starting page. When the split point exists in the page batch, and the last split point is located at the bottom of the page, the next page of that page is taken as the starting page. Repeat the process of selecting page batches again starting from the aforementioned starting page and generating new starting pages until the entire target source document is divided into page batches; Within each page batch, the corresponding page is segmented at the split point to form the fragment document.

9. A vision-based document parsing system according to claim 6, characterized in that, The third artificial intelligence model is configured to: extract pagination page numbers from the metadata and perform text duplication detection on the same document segment; When the pagination page number is abnormal or there is an abnormality in the text repetition, it is determined that the parsed segment is abnormal.

Images