Document segmentation method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and financial science and technology, and relates to a document segmentation method, which comprises the following steps: carrying out style feature recognition and title text recognition on a to-be-segmented document to obtain a title style result and a title text result, and then carrying out recognition verification on the title style result and the title text result through a multi-modal title recognition large model to obtain a segmented document. And obtaining a final title recognition result, constructing tree hierarchical representation based on the title recognition result to obtain a document chapter structure, and segmenting the to-be-segmented document according to the document chapter structure. The invention further provides a document segmentation device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the to-be-segmented document can be stored in a block chain. According to the method and the device, the document is segmented in a chapter form, the semantic integrity of segmentation is ensured, and key information is effectively extracted and structured.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to a document segmentation method, apparatus, computer equipment and storage medium. Background Art

[0002] In the era of information explosion, efficiently processing and utilizing massive data has become a key issue that needs to be solved in various industries. With the rapid development of artificial intelligence technology, especially the continuous progress in the field of natural language processing (NLP), RAG (Retrieval-Augmented Generation) technology is an advanced method that combines information retrieval and text generation. It retrieves relevant information from a large amount of data and uses this information to enhance the answering ability of large models. Among them, document slicing is a key step in RAG technology, which involves breaking down large documents or data sets into smaller, easy-to-manage and retrievable fragments, which is crucial to the efficiency and quality of subsequent information retrieval and text generation.

[0003] A common document parsing solution is to directly extract all text information in a document, simply segment the text content into chunks based on spacing or blank spaces or fixed window sizes, and convert these chunks into vectors and store them in a database for semantic retrieval. However, these methods have the following problems in practical applications:

[0004] 1) Loss of semantic integrity: Since the segmentation strategy does not consider the inherent logical structure of the document content, such as chapters and paragraphs, the same topic or logically coherent content is segmented into different text blocks. This semantic break not only destroys the integrity of the original document, but also seriously affects the accuracy and effectiveness of the subsequent semantic vector representation, thereby reducing the accuracy and relevance of the retrieval results.

[0005] 2) Ambiguity in multi-document retrieval: In scenarios involving multi-document retrieval, due to the lack of effective document titles or structured information, matching based solely on the similarity of text content often leads to ambiguity. That is, similar text content between different documents may point to completely different contexts or topics, thus misleading the retrieval system and affecting the accuracy of the final results.

[0006] 3) Loss of format information: For documents that are not in plain text format (such as documents containing rich media elements such as pictures, tables, formulas, etc.), traditional segmentation methods often cannot effectively retain these important information, resulting in format confusion or complete loss. This not only limits the richness and diversity of text generation, but also weakens the practicality and readability of retrieval results.

[0007] Therefore, in view of the above problems, a more intelligent and efficient document segmentation method is urgently needed to solve the deficiencies in the existing technology. It can effectively extract and structure key information while maintaining the semantic integrity of the document content, providing solid data support for RAG technology and other NLP-based applications. Therefore, this patent application aims to propose an innovative document slicing and parsing method to improve the overall performance of information retrieval and text generation. Summary of the invention

[0008] The purpose of the embodiments of the present application is to propose a document segmentation method, apparatus, computer equipment and storage medium to solve the technical problems of lack of semantic integrity, ambiguity in multi-document retrieval and inability to retain document format information in the prior art document segmentation.

[0009] In order to solve the above technical problems, the present application embodiment provides a document segmentation method, which adopts the following technical solution:

[0010] Obtaining a document to be segmented, performing style feature recognition on the document to be segmented, and obtaining a title style result;

[0011] Performing title text recognition on the document to be segmented to obtain a title text result;

[0012] Through the trained multimodal title recognition large model, the title style result and the title text result are recognized and verified to obtain the title recognition result;

[0013] Extracting titles and corresponding title hierarchical relationships in the title recognition results, constructing a tree-like hierarchical representation based on the titles and the title hierarchical relationships, and obtaining a document chapter structure;

[0014] The document to be segmented is segmented according to the document chapter structure to obtain a document segmentation result.

[0015] In order to solve the above technical problems, the embodiment of the present application further provides a document segmentation device, which adopts the following technical solution:

[0016] The style recognition module is used to obtain the document to be segmented, perform style feature recognition on the document to be segmented, and obtain a title style result;

[0017] A text recognition module is used to perform title text recognition on the document to be segmented to obtain a title text result;

[0018] A multimodal verification module is used to perform recognition verification on the title style result and the title text result through the trained multimodal title recognition large model to obtain a title recognition result;

[0019] A structure construction module, used to extract the titles and corresponding title hierarchical relationships in the title recognition results, construct a tree-like hierarchical representation based on the titles and the title hierarchical relationships, and obtain a document chapter structure;

[0020] The segmentation module is used to segment the document to be segmented according to the document chapter structure to obtain a document segmentation result.

[0021] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0022] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the document segmentation method described above when executing the computer-readable instructions.

[0023] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0024] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the document segmentation method described above are implemented.

[0025] Compared with the prior art, this application has the following beneficial effects:

[0026] The present application provides a document segmentation method, which obtains title style results and title text results by performing style feature recognition and title text recognition on the document to be segmented, and then performs recognition and verification on the title style results and title text results through a multimodal title recognition large model to obtain the final title recognition result, which can achieve effective fusion of style and text features, thereby improving the accuracy of title recognition; construct a tree-like hierarchical representation based on the title recognition result to obtain the document chapter structure, and segment the document to be segmented according to the document chapter structure, thereby achieving the segmentation of the document in the form of chapters, ensuring the semantic integrity of the segmentation, effectively extracting and structuring key information, avoiding ambiguity, and obtaining a text content order consistent with the reading order, thereby improving the practicality and readability of the segmented text. In addition, the present application is particularly suitable for situations where there are many document styles and a small amount of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1is an exemplary system architecture diagram to which the present application may be applied;

[0029] Figure 2 is a flowchart of an embodiment of a document segmentation method according to the present application;

[0030] Figure 3 is a structural schematic diagram of an embodiment of a document segmentation device according to the present application;

[0031] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0036] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0037] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0038] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0039] It should be noted that the document segmentation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the document segmentation device is generally arranged in the server / terminal device.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0041] Continue to refer Figure 2 , shows a flowchart of an embodiment of a document segmentation method according to the present application, comprising the following steps:

[0042] Step S201, obtaining a document to be segmented, performing style feature recognition on the document to be segmented, and obtaining a title style result.

[0043] The documents to be segmented are documents in the financial field that need to be processed, including but not limited to insurance contracts, claim applications, insurance applications, business reports, risk assessment reports, financial statements, etc. The formats of the documents to be segmented include PDF, Word, HTML, DOC, DOCX, XLS, etc., which are not limited here.

[0044] In this embodiment, through style feature recognition, the spatial layout, font type, font size, bold italic, color, alignment, line spacing, first line indent, position and other information of the document to be segmented are analyzed and extracted. By analyzing the layout features, different elements in the document, such as text, title, picture, table, etc., can be distinguished.

[0045] Based on the results of style feature recognition, the title part in the document can be further identified and extracted, and then the title style results can be obtained. Titles usually have specific format features, such as larger fonts, bold, centered or specific colors.

[0046] In this embodiment, the document segmentation method is executed on the electronic device (eg Figure 1 The server / terminal device shown in the figure can obtain the document to be segmented through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0047] It should be emphasized that in order to further ensure the privacy and security of the documents to be segmented, the above-mentioned documents to be segmented can also be stored in a node of a blockchain.

[0048] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0049] In some optional implementations, the step of performing style feature recognition on the document to be segmented to obtain a title style result includes:

[0050] Inputting the document to be segmented into a layout style recognition model, the layout style recognition model includes a text recognition layer, a style recognition layer and a style output layer;

[0051] Each line of text in the document to be segmented is located through the text recognition layer to obtain the text line positioning box feature;

[0052] The font style is extracted from the text line positioning box feature through the style recognition layer to obtain the font style feature;

[0053] The text line positioning box feature and the font style feature are fused to obtain the text style feature;

[0054] The text style features are classified and predicted through the style output layer to obtain the title style results.

[0055] In this embodiment, the line text of the document to be segmented is positioned by the PSEnet algorithm to obtain a text line positioning frame. Among them, the PSEnet algorithm is implemented by the PSEnet network. The architecture of the PSEnet network is resnet+FPN (FeaturePyramid Networks). Features are extracted by resnet, and features of different layers are taken and sent to FPN for feature fusion to achieve a progressive expansion algorithm of the segmentation result. The characteristic of the PSEnet algorithm is that it can locate text of any shape. When the PSEnet algorithm locates text, it first predicts different kernels for each text line. These kernels generally have the same shape and center as the text line and do not exceed the size of the original text line. After that, the BFS-based expansion algorithm is used for these kernels. First, start from the smallest scale, gradually enlarge, and finally expand to the size of the original text to obtain the cell-located text line, that is, the text line positioning frame.

[0056] The PSEnet algorithm can also obtain the position information of the text line positioning box, where the position information includes the four vertex coordinates of the text line positioning box, the upper left corner coordinate (x1, y1), the lower left corner coordinate (x2, y2), the upper right corner coordinate (x3, y3) and the lower right corner coordinate (x4, y4).

[0057] In this embodiment, the text recognition layer uses the PSEnet algorithm to locate the text lines of the document to be segmented. After obtaining the text line positioning box, the corresponding title judgment formula is called according to the document type of the document to be segmented. According to the position information of the text line positioning box, the title judgment formula is used to calculate the judgment result, and the candidate title text line positioning box is determined according to the judgment result, the title position information of the candidate title text line positioning box is obtained, and the text line positioning box feature containing the title position information is output.

[0058] The title judgment formula is a calculation formula based on the title position in a large amount of document data of each document type. For example, the title in an insurance document is usually centered and bold, so the title judgment formula is a centering judgment formula, which is as follows:

[0059]

[0060] Among them, page_width is the width of the page, x0 and x1 are the left and right border points of the text box respectively.

[0061] The text line positioning box features are input into the style recognition layer to extract the font style. The style recognition layer is a visual feature extraction layer. Traditional algorithms such as scale-invariant feature transform (SIFT), histogram of oriented gradients (HOG), local binary patterns (LBP), etc. can be used, or deep learning algorithms such as convolutional neural networks, self-decoders, etc. can be used.

[0062] The text line positioning box features and font style features are spliced ​​to obtain the fused text style features, and the text style features are mapped to the title style through the style output layer to obtain the title style results, where the title style results include the title type and title position information; the title type is a hierarchical title, including a first-level title, a second-level title, a third-level title, etc.

[0063] In some optional implementations of this embodiment, the format of the document to be segmented is determined. If the document to be segmented is not in PDF format, the document to be segmented is converted into a PDF document; the PDF document is read through PyMuPDF to obtain the border information and font style information of each line of text; by analyzing a large amount of document data of the same type, the centered and bold text is judged as a candidate title based on the border information and font style information, and the recognition result is output.

[0064] By identifying the document style, the document processing efficiency can be improved, and the key information of the title in the document can be extracted, which helps to enhance the document's structuring capabilities and improve the accuracy of information extraction. At the same time, it can be compatible with multiple document formats, enhancing the versatility and practicality of the system.

[0065] Step S202, performing title text recognition on the document to be segmented to obtain a title text result.

[0066] In this embodiment, the document content of the document to be segmented is recognized, and the title in the document is recognized. Specifically, text recognition is performed on the document to be segmented to obtain text content; the text content is input into a trained title text recognition model, wherein the title text recognition model includes a text input layer, an embedding layer, a decoding layer, and a text output layer; the text recognition result is preprocessed through the text input layer to obtain a standard input text; the standard input text is input into the embedding layer for vector conversion to obtain a text embedding vector; the text embedding vector is subjected to attention calculation through the decoding layer to obtain a title text representation; the title text representation is input into the text output layer for output to obtain a title text result.

[0067] Convert the document to be segmented into images, and perform text recognition on the document content through OCR (optical character recognition) technology to obtain the text content, call the large model, take the text content as input, perform semantic analysis on the text content through the written prompt, identify whether the text content is a title, and determine the level of the recognized title, and then output the title text result in markdown format, where the output title text result is marked with a level mark, such as # for a first-level title, ## for a second-level title, and so on.

[0068] In this embodiment, the backbone network of the title text recognition large model is the Qwen-72B large model, including a text input layer, an embedding layer, a decoding layer and a text output layer, wherein the text input layer is used to receive text content and convert the input text content (such as characters, words or sentences) into a format that can be processed by the model. At the same time, the text content is preprocessed, including removing punctuation marks, converting to lowercase, segmenting words, etc., so that subsequent layers can process text data more efficiently; the embedding layer is used to convert the standard input text output by the text input layer into text embedding vectors, which can capture the semantic relationship and contextual information between words.

[0069] In some optional implementations, the encoding layer is composed of multiple stacked Transformer decoders, and the output of the previous decoder is used as the input of the next decoder. Each decoder includes a mixed attention layer, an attention fusion layer, and a feedforward network layer.

[0070] Specifically, the text embedding vector is input into the decoder, and the local attention calculation and multi-head attention calculation are performed on the text embedding vector respectively through the mixed attention layer to obtain the text local attention feature and the text multi-head attention feature; the text local attention feature and the text multi-head attention feature are fused through the attention fusion layer to obtain the attention fusion feature; the attention fusion feature is input into the feedforward network layer for text prediction, and the title text representation is output.

[0071] The hybrid attention layer includes two branches, one is the multi-head attention branch and the other is the local attention branch. The text embedding vector is respectively input into the multi-head attention branch and the local attention branch for attention calculation to capture the key information and contextual semantics in the text. In the local attention calculation process, the window size needs to be defined. The window size is a hyperparameter and can be adjusted during the training process. The local attention is calculated through the window. The outputs of the two branches are fused through the attention fusion layer, and the fused attention fusion features are connected and enhanced through the feedforward network layer to generate the title text representation.

[0072] In this embodiment, the input of the next decoder is the output of the feedforward network layer of the previous decoder and the text embedding vector. After all decoders capture the key information and contextual semantics in the text, the output of the feedforward network layer of the last decoder is the final predicted title text representation.

[0073] By performing attention calculation on the text embedding vector through the decoder of the decoding layer, attention information can be transferred between decoders, deep features in the text can be captured, the representation capability of the features can be enriched, the generalization ability of the model can be improved, and the model can understand the input text more comprehensively, thus improving the model's ability to understand and process complex data structures.

[0074] In some optional implementations, residual connections and layer normalization are performed after each attention fusion layer and feed-forward network layer, which can help gradient flow and reduce degradation problems during training.

[0075] By calling the trained title text recognition large model for title text recognition, the contextual information of the text can be fully considered, so as to more accurately identify the title of the text, adapt to complex text scenarios, significantly improve the efficiency and accuracy of text processing, and thus improve the accuracy of recognizing text titles.

[0076] Step S203, using the trained multimodal title recognition large model, the title style result and the title text result are recognized and verified to obtain the title recognition result.

[0077] In this embodiment, a multimodal large model is used to perform a secondary judgment and verification on the recognized title style results and title text results. By comprehensively analyzing the image features and text features of the title, the probability that the recognized title is a real title is calculated. If the probability exceeds a preset threshold, it is determined to be the real title of the document, otherwise it is judged as non-title text.

[0078] In some optional implementations, the multimodal title recognition large model completed through training performs recognition verification on the title style result and the title text result to obtain the title recognition result, and the step includes:

[0079] Inputting the title style result and the title text result into a multimodal title recognition large model, wherein the multimodal title recognition large model includes a visual feature extraction layer, a text feature extraction layer, a cross-modal alignment layer and a result output layer;

[0080] The visual feature extraction layer extracts features from the title style results to obtain style visual features;

[0081] The text feature extraction layer performs semantic analysis on the title text results to obtain text semantic features;

[0082] Input the style visual features and text semantic features into the cross-modal alignment layer for feature fusion to obtain text style fusion features;

[0083] The text style fusion features are inferred and predicted through the result output layer to obtain the title recognition result containing the title level information.

[0084] In this embodiment, the backbone network of the multimodal title recognition model is the interVL model. The visual feature extraction layer extracts visual features of the title style results, and the visual feature extraction layer uses a fine-tuned ViT (Vision Transformer) model for feature extraction. The text feature extraction layer can use pre-trained language models such as BERT, RoBERTa, etc. to extract text features from the title text results. The style visual features and text semantic features are fused through the cross-modal alignment layer for multimodal features, and reasoning is performed based on the fused visual features and text features to output the final title recognition result.

[0085] Through the multimodal title recognition large model, high-precision recognition of document titles and complete extraction of semantic features can be achieved, further ensuring the accuracy of title extraction.

[0086] Step S204, extracting titles and corresponding title hierarchical relationships in the title recognition results, constructing a tree-like hierarchical representation based on the titles and title hierarchical relationships, and obtaining a document chapter structure.

[0087] Among them, the title recognition result includes the title and the title hierarchy relationship corresponding to the title, and the title hierarchy relationship includes a first-level title, a second-level title, a third-level title, etc.

[0088] In this embodiment, the steps of constructing a tree-like hierarchical representation based on titles and title hierarchical relationships to obtain a document chapter structure include:

[0089] According to the page order of the document to be segmented, all the titles in the document to be segmented are spliced ​​according to the title hierarchy relationship to obtain an initial title hierarchy structure;

[0090] Using graph algorithm, we take titles as nodes, add directed edges according to the semantic relevance between titles, and construct a directed acyclic graph of titles;

[0091] Through the minimum spanning tree algorithm, the minimum spanning tree with the root node as the source is found in the title directed acyclic graph as the optimized title hierarchy structure;

[0092] The document chapter structure of the document to be segmented is extracted according to the optimized title hierarchy structure.

[0093] When constructing the tree-like hierarchical representation of the title, the titles of all single-page documents are spliced ​​according to the title hierarchy in the order of the document pages. Specifically, a recursive algorithm is used to construct a title tree from bottom to top, in which each node represents a title, the parent node represents the previous level title, and the leaf node represents the bottom-level title. For example, for the titles that appear in sequence, such as "#Introduction", "##Research Background", "##Problem Definition", and "#Related Work", a two-layer tree structure can be constructed: {Introduction: {Research Background, Problem Definition}, Related Work}.

[0094] Using a graph algorithm, taking titles as nodes, adding directed edges according to the semantic relevance between titles, constructing a title directed acyclic graph (DAG), and using a minimum spanning tree algorithm such as Edmonds' algorithm, finding a minimum spanning tree with the root node as the source in the title directed acyclic graph as the optimized title hierarchy. For example, in the initial title hierarchy, there may be a lack of necessary hierarchical relationship between "Research Background" and "Problem Definition" under "Introduction". By calculating the semantic similarity between titles, it is found that "Research Background" has a higher similarity with "Related Work", so it can be removed from the "Introduction" subnode and used as a subnode of "Related Work".

[0095] The title hierarchy structure is a tree structure, which can reflect the chapter structure of the document. The chapter structure of the document can be directly obtained from the optimized title hierarchy structure.

[0096] By constructing a tree-like title hierarchy, a complete representation of the document structure and content is formed, which effectively retains the key structural information of the document and ensures the semantic integrity of subsequent document segmentation.

[0097] Step S205, segmenting the document to be segmented according to the document chapter structure to obtain a document segmentation result.

[0098] In this embodiment, the document content to be segmented is segmented according to the document chapter structure, and the content under each title is extracted to form document segments with titles as units. The document article structure and the corresponding document segments are saved in a structured data format to facilitate subsequent document retrieval and information extraction.

[0099] In some optional implementations, by calculating the semantic similarity of document fragments, semantically related fragments are merged to reduce the number of document fragments and improve the organization and readability of the document.

[0100] This application obtains title style results and title text results by performing style feature recognition and title text recognition on the document to be segmented, and then uses a multimodal title recognition large model to recognize and verify the title style results and title text results to obtain the final title recognition result, which can achieve effective fusion of style and text features, thereby improving the accuracy of title recognition; based on the title recognition result, a tree-like hierarchical representation is constructed to obtain the document chapter structure, and the document to be segmented is segmented according to the document chapter structure, thereby achieving the segmentation of the document in the form of chapters, ensuring the semantic integrity of the segmentation, effectively extracting and structuring key information, avoiding ambiguity, and obtaining a text content order consistent with the reading order, thereby improving the practicality and readability of the segmented text. In addition, this application is particularly suitable for situations where there are many document styles and a small amount of data.

[0101] In some optional implementations, after the step of segmenting the document to be segmented according to the document chapter structure to obtain the document segmentation result, the method further includes:

[0102] Obtain the text content of each chapter according to the document segmentation results, and convert the text content of each chapter into a chapter semantic vector;

[0103] Calculate the chapter similarity between each chapter text through the chapter semantic vector, and build a chapter relationship map based on the chapter similarity;

[0104] Generate document structured summary and document knowledge graph based on the document chapter structure and chapter relationship graph.

[0105] Specifically, according to the document chapter structure, the document content is divided into several chapters, each chapter corresponds to a title, and the chapter text content corresponding to each chapter is extracted. The pre-trained language model is used to convert the chapter text content of each chapter into a semantic vector representation to obtain a chapter semantic vector. The chapter relationship graph is constructed by calculating the cosine similarity between the chapter vectors. Combining the title structure and the chapter relationship graph, a document structured summary is generated through the seq2seq model. The summary contains the main content of the document and the logical relationship between chapters. Key information, such as entities, relationships, etc., is extracted from the chapter content, and a document knowledge graph is constructed in combination with the domain knowledge base. The document structured summary and knowledge graph are used as the final output of the document structuring processing for subsequent information retrieval and knowledge reasoning tasks.

[0106] This application can facilitate subsequent information retrieval and knowledge reasoning tasks by constructing document structured summaries and document knowledge graphs, and further improve the accuracy and efficiency of title recognition.

[0107] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0108] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0110] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0111] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a document segmentation device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0112] like Figure 3As shown, the document segmentation device 300 described in this embodiment includes: a style recognition module 301, a text recognition module 302, a multi-modal verification module 303, a structure construction module 304 and a segmentation module 305. Among them:

[0113] The style recognition module 301 is used to obtain the document to be segmented, perform style feature recognition on the document to be segmented, and obtain a title style result;

[0114] The text recognition module 302 is used to perform title text recognition on the document to be segmented to obtain a title text result;

[0115] The multimodal verification module 303 is used to perform recognition verification on the title style result and the title text result through the trained multimodal title recognition large model to obtain the title recognition result;

[0116] The structure construction module 304 is used to extract the titles and the corresponding title hierarchical relationships in the title recognition results, and construct a tree-like hierarchical representation based on the titles and the title hierarchical relationships to obtain a document chapter structure;

[0117] The segmentation module 305 is used to segment the document to be segmented according to the document chapter structure to obtain a document segmentation result.

[0118] It should be emphasized that in order to further ensure the privacy and security of the documents to be segmented, the above-mentioned documents to be segmented can also be stored in a node of a blockchain.

[0119] Based on the above-mentioned document segmentation device 300, by performing style feature recognition and title text recognition on the document to be segmented, the title style result and title text result are obtained, and then the title style result and title text result are recognized and verified by the multimodal title recognition large model to obtain the final title recognition result, which can realize the effective fusion of style and text features, thereby improving the accuracy of title recognition; based on the title recognition result, a tree-like hierarchical representation is constructed to obtain the document chapter structure, and the document to be segmented is segmented according to the document chapter structure, so as to realize the segmentation of the document in the form of chapters, ensure the semantic integrity of the segmentation, effectively extract and structure key information, avoid ambiguity, and obtain the text content order consistent with the reading order, thereby improving the practicality and readability of the segmented text. In addition, the present application is particularly suitable for situations where there are many document styles and a small amount of data.

[0120] In some optional implementations, the style identification module 301 includes:

[0121] An input submodule, used for inputting the document to be segmented into a layout style recognition model, wherein the layout style recognition model includes a text recognition layer, a style recognition layer and a style output layer;

[0122] A text positioning submodule, used for positioning each line of text in the document to be segmented through the text recognition layer to obtain text line positioning box features;

[0123] A style feature extraction submodule, used to extract the font style of the text line positioning box feature through the style recognition layer to obtain the font style feature;

[0124] A fusion submodule, used for fusing the text line positioning box feature and the text font style feature to obtain a text style feature;

[0125] The style prediction submodule is used to classify and predict the text style features through the style output layer to obtain a title style result.

[0126] By identifying the document style, the document processing efficiency can be improved, and the key information of the title in the document can be extracted, which helps to enhance the document's structuring capabilities and improve the accuracy of information extraction. At the same time, it can be compatible with multiple document formats, enhancing the versatility and practicality of the system.

[0127] In some optional implementations, the text recognition module 302 includes:

[0128] A text recognition submodule is used to perform text recognition on the document to be segmented to obtain text content;

[0129] A text input submodule, used for inputting the text content into a trained title text recognition model, wherein the title text recognition model includes a text input layer, an embedding layer, a decoding layer and a text output layer;

[0130] A preprocessing submodule, used for preprocessing the text recognition result through the text input layer to obtain a standard input text;

[0131] An embedding submodule, used for inputting the standard input text into the embedding layer for vector conversion to obtain a text embedding vector;

[0132] An attention calculation submodule, used for performing attention calculation on the text embedding vector through the decoding layer to obtain a title text representation;

[0133] The output submodule is used to input the title text representation into the text output layer for output to obtain a title text result.

[0134] By calling the trained title text recognition large model for title text recognition, the contextual information of the text can be fully considered, so as to more accurately identify the title of the text, adapt to complex text scenarios, significantly improve the efficiency and accuracy of text processing, and thus improve the accuracy of recognizing text titles.

[0135] In some optional implementations of this embodiment, the decoding layer includes a plurality of decoders connected in sequence, and the attention calculation submodule includes:

[0136] An input unit, used to input the text embedding vector into the decoder, each of the decoders comprising a hybrid attention layer, an attention fusion layer, and a feedforward network layer;

[0137] An attention mechanism unit, used to perform local attention calculation and multi-head attention calculation on the text embedding vector through the hybrid attention layer to obtain text local attention features and text multi-head attention features;

[0138] A fusion unit, used for fusing the text local attention feature and the text multi-head attention feature through the attention fusion layer to obtain an attention fusion feature;

[0139] The feedforward unit is used to input the attention fusion feature into the feedforward network layer for text prediction and output the title text representation.

[0140] By performing attention calculation on the text embedding vector through the decoder of the decoding layer, attention information can be transferred between decoders, deep features in the text can be captured, the representation capability of the features can be enriched, the generalization ability of the model can be improved, and the model can understand the input text more comprehensively, thus improving the model's ability to understand and process complex data structures.

[0141] In some optional implementations, the multimodal verification module 303 includes:

[0142] A multimodal input submodule, used for inputting the title style result and the title text result into the multimodal title recognition large model, wherein the multimodal title recognition large model includes a visual feature extraction layer, a text feature extraction layer, a cross-modal alignment layer and a result output layer;

[0143] A visual submodule, used for extracting features from the title style result through the visual feature extraction layer to obtain style visual features;

[0144] A language submodule, used for performing semantic analysis on the title text result through the text feature extraction layer to obtain text semantic features;

[0145] An alignment fusion submodule, used for inputting the style visual features and the text semantic features into the cross-modal alignment layer for feature fusion to obtain text style fusion features;

[0146] The fusion prediction submodule is used to perform inference prediction on the text style fusion feature through the result output layer to obtain a title recognition result containing title level information.

[0147] Through the multimodal title recognition large model, high-precision recognition of document titles and complete extraction of semantic features can be achieved, further ensuring the accuracy of title extraction.

[0148] In some optional implementations, the structure building module 304 includes:

[0149] The structure construction submodule is used to splice all the titles in the document to be segmented according to the page order of the document to be segmented according to the title hierarchy relationship to obtain an initial title hierarchy structure;

[0150] A graph submodule, for adopting a graph algorithm, taking the titles as nodes, adding directed edges according to the semantic relevance between the titles, and constructing a directed acyclic graph of titles;

[0151] An optimization submodule, used for searching, by a minimum spanning tree algorithm, a minimum spanning tree with a root node as a source in the title directed acyclic graph as an optimized title hierarchical structure;

[0152] The extraction submodule is used to extract the document chapter structure of the document to be segmented according to the optimized title hierarchy structure.

[0153] By constructing a tree-like title hierarchy, a complete representation of the document structure and content is formed, which effectively retains the key structural information of the document and ensures the semantic integrity of subsequent document segmentation.

[0154] In some optional implementations, the document segmentation device 300 further includes a graph construction module, which is used to:

[0155] Acquire the text content of each chapter according to the document segmentation result, and convert each of the chapter text content into a chapter semantic vector;

[0156] Calculating the chapter similarity between each of the chapter texts through the chapter semantic vector, and constructing a chapter relationship map based on the chapter similarity;

[0157] A document structured summary and a document knowledge graph are generated according to the document chapter structure and the chapter relationship graph.

[0158] By constructing document structured summaries and document knowledge graphs, subsequent information retrieval and knowledge reasoning tasks can be facilitated, further improving the accuracy and efficiency of title recognition.

[0159] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0160] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0161] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0162] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the document segmentation method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0163] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as running the computer-readable instructions of the document segmentation method.

[0164] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0165] By performing style feature recognition and title text recognition on the document to be segmented, the title style result and title text result are obtained, and then the title style result and title text result are recognized and verified by a large multimodal title recognition model to obtain the final title recognition result, which can achieve the effective fusion of style and text features, thereby improving the accuracy of title recognition; a tree-like hierarchical representation is constructed based on the title recognition result to obtain the document chapter structure, and the document to be segmented is segmented according to the document chapter structure, thereby achieving the segmentation of the document in the form of chapters, ensuring the semantic integrity of the segmentation, effectively extracting and structuring key information, avoiding ambiguity, and obtaining a text content order consistent with the reading order, thereby improving the practicality and readability of the segmented text. In addition, the present application is particularly suitable for situations where there are many document styles and a small amount of data.

[0166] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the document segmentation method as described above.

[0167] By performing style feature recognition and title text recognition on the document to be segmented, the title style result and title text result are obtained, and then the title style result and title text result are recognized and verified by a large multimodal title recognition model to obtain the final title recognition result, which can achieve the effective fusion of style and text features, thereby improving the accuracy of title recognition; a tree-like hierarchical representation is constructed based on the title recognition result to obtain the document chapter structure, and the document to be segmented is segmented according to the document chapter structure, thereby achieving the segmentation of the document in the form of chapters, ensuring the semantic integrity of the segmentation, effectively extracting and structuring key information, avoiding ambiguity, and obtaining a text content order consistent with the reading order, thereby improving the practicality and readability of the segmented text. In addition, the present application is particularly suitable for situations where there are many document styles and a small amount of data.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0169] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A document segmentation method, characterized in that: The steps include: Obtaining a document to be segmented, performing style feature recognition on the document to be segmented, and obtaining a title style result; Performing title text recognition on the document to be segmented to obtain a title text result; Through the trained multimodal title recognition large model, the title style result and the title text result are recognized and verified to obtain the title recognition result; Extracting titles and corresponding title hierarchical relationships in the title recognition results, constructing a tree-like hierarchical representation based on the titles and the title hierarchical relationships, and obtaining a document chapter structure; The document to be segmented is segmented according to the document chapter structure to obtain a document segmentation result.

2. The document segmentation method according to claim 1, characterized in that: The step of performing style feature recognition on the document to be segmented to obtain a title style result comprises: Inputting the document to be segmented into a layout style recognition model, wherein the layout style recognition model includes a text recognition layer, a style recognition layer and a style output layer; Positioning each line of text in the document to be segmented by the text recognition layer to obtain text line positioning box features; Extracting the font style of the text line positioning frame feature through the style recognition layer to obtain the font style feature; Fusing the text line positioning box feature and the font style feature to obtain a text style feature; The text style features are classified and predicted through the style output layer to obtain a title style result.

3. The document segmentation method according to claim 1, characterized in that: The step of performing title text recognition on the document to be segmented to obtain the title text result comprises: Performing text recognition on the document to be segmented to obtain text content; Inputting the text content into a trained title text recognition model, wherein the title text recognition model includes a text input layer, an embedding layer, a decoding layer, and a text output layer; Preprocessing the text recognition result through the text input layer to obtain a standard input text; Inputting the standard input text into the embedding layer for vector conversion to obtain a text embedding vector; Performing attention calculation on the text embedding vector through the decoding layer to obtain a title text representation; The title text representation is input into the text output layer for output to obtain a title text result.

4. The document segmentation method according to claim 3, characterized in that: The decoding layer includes a plurality of decoders connected in sequence, and the step of performing attention calculation on the text embedding vector through the decoding layer to obtain the title text representation includes: Inputting the text embedding vector into the decoder, each of the decoders includes a mixed attention layer, an attention fusion layer, and a feedforward network layer; The mixed attention layer performs local attention calculation and multi-head attention calculation on the text embedding vector respectively to obtain text local attention features and text multi-head attention features; The text local attention feature and the text multi-head attention feature are fused through the attention fusion layer to obtain an attention fusion feature; The attention fusion feature is input into the feed-forward network layer for text prediction, and the title text representation is output.

5. The document segmentation method according to claim 1, characterized in that: The multimodal title recognition large model completed through training performs recognition verification on the title style result and the title text result to obtain the title recognition result, which includes: Inputting the title style result and the title text result into the multimodal title recognition large model, wherein the multimodal title recognition large model includes a visual feature extraction layer, a text feature extraction layer, a cross-modal alignment layer and a result output layer; Extracting features from the title style result through the visual feature extraction layer to obtain style visual features; Performing semantic analysis on the title text result through the text feature extraction layer to obtain text semantic features; Inputting the style visual features and the text semantic features into the cross-modal alignment layer for feature fusion to obtain text style fusion features; The text style fusion feature is inferred and predicted through the result output layer to obtain a title recognition result containing title level information.

6. The document segmentation method according to claim 1, characterized in that: The step of constructing a tree-like hierarchical representation based on the titles and the title hierarchical relationships to obtain a document chapter structure comprises: According to the page order of the document to be segmented, all the titles in the document to be segmented are spliced ​​according to the title hierarchy relationship to obtain an initial title hierarchy structure; Using a graph algorithm, taking the titles as nodes, adding directed edges according to the semantic relevance between the titles, and constructing a title directed acyclic graph; By using a minimum spanning tree algorithm, a minimum spanning tree with a root node as a source is searched in the title directed acyclic graph as an optimized title hierarchical structure; The document chapter structure of the document to be segmented is extracted according to the optimized title hierarchy structure.

7. The document segmentation method according to claim 1, characterized in that: After the step of segmenting the document to be segmented according to the document chapter structure to obtain the document segmentation result, the step further includes: Acquire the text content of each chapter according to the document segmentation result, and convert each of the chapter text content into a chapter semantic vector; Calculating the chapter similarity between each of the chapter texts through the chapter semantic vector, and constructing a chapter relationship map based on the chapter similarity; A document structured summary and a document knowledge graph are generated according to the document chapter structure and the chapter relationship graph.

8. A document segmentation device, characterized in that: include: The style recognition module is used to obtain the document to be segmented, perform style feature recognition on the document to be segmented, and obtain a title style result; A text recognition module is used to perform title text recognition on the document to be segmented to obtain a title text result; A multimodal verification module, used to perform recognition verification on the title style result and the title text result through the trained multimodal title recognition large model to obtain a title recognition result; A structure construction module, used to extract the titles and corresponding title hierarchical relationships in the title recognition results, construct a tree-like hierarchical representation based on the titles and the title hierarchical relationships, and obtain a document chapter structure; The segmentation module is used to segment the document to be segmented according to the document chapter structure to obtain a document segmentation result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the document segmentation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the document segmentation method according to any one of claims 1 to 7 are implemented.

Images