Document analysis and index construction method

Abstract

The invention discloses a document analysis and index construction method. The method comprises the following steps of layout detection, table image analysis and extraction, page recovery, content splitting and index construction and storage. According to layout detection, a document layout analysis model is used for carrying out area detection and classification on images, tables, titles and text elements of pdf document pages; the table image analysis extraction is to extract and process formats and contents of tables and images existing in each page of the pdf obtained based on layout detection; according to the method for analyzing the document and constructing the index, intelligent analysis is provided, texts, tables and images in the pdf document can be accurately extracted, and perfect index elements can be constructed.

Description

Technical Field

[0001] The present invention relates to the technical field of document parsing and indexing, and more specifically, to a document parsing and indexing construction method. Background Art

[0002] At present, the general large language model (LLM) has made great progress in knowledge question answering, but it is still powerless in professional fields, because a lot of data in professional fields will not be made public. General LLM has not learned professional data and naturally cannot make accurate answers to professional knowledge. One idea is to feed these professional data to LLM for fine-tuning, but the technical and cost requirements are often too high. The retrieval augmentation generation (RAG) system is another way to solve professional question answering. After the user's original question, the private domain data related to it is added to the question, and the LLM analyzes and summarizes it. Providing more accurate information to LLM through retrieval enhancement can improve the answer effect, such as Figure 1 shown.

[0003] Among them, document processing is the prerequisite for knowledge base retrieval, and it is necessary to convert various private domain documents into retrievable data offline. Although RAG is suitable for scenarios in professional fields where knowledge is frequently updated, in actual scenarios, most professional documents are stored in unstructured data such as pdf and doc. They have titles, paragraphs, tables, pictures and other elements, which are easy for humans to read, but not suitable for computer retrieval and processing. Document parsing is the first step of the RAG system. Only by improving the quality and speed of document parsing can the RAG enterprise be truly implemented in actual scenarios. The better the document parsing, the more accurate the retrieved knowledge will be, and ultimately the LLM will answer more accurately and less prone to hallucinations.

[0004] When parsing word and pdf documents, the difficulty of parsing is different because they have different encoding formats. Word documents tend to be edited and follow the Office Open XML standard. The underlying XML saves data and has concepts such as titles, paragraphs, and tables, but does not contain the concept of pages and positions. The final display position of each element of the document is determined by the actual rendering engine. To parse a word document, you only need to read the underlying XML file according to the standard.

[0005] However, PDF documents are more suitable for reading and printing, and their display effects are consistent on different terminals. Compared with word, PDF has the concept of pages and positions. Because it does not need to be edited, there are no concepts such as titles, paragraphs, and tables in PDF. For example, titles are just large bold text, and tables are just aligned lines and text. In addition to extracting plain text, parsing PDF documents also requires additional operations such as layout recovery, table extraction, and image extraction. There are some open source python tools for PDF parsing, as shown in the following table 1: Table 1 Tool Name Features advantage shortcoming PDFMiner Designed specifically for extracting PDF text content, it provides underlying PDF parsing capabilities and supports extracting text, images, and other information. Powerful parsing capabilities, support for complex PDF structures, and strong customizability The API is relatively complex to use and its performance may not be as good as other tools PyPDF A lightweight PDF processing library that supports basic reading, writing, merging, splitting, encryption, etc. Suitable for simple PDF processing tasks The functionality is relatively limited and not good at complex text extraction and layout analysis PyMuPDF Provides fast rendering and manipulation of PDF, XPS, EPUB and other document formats Excellent performance, comprehensive functions, detailed API documentation The underlying language is C, which is not easy to customize. PDFPlumber Based on PDFMiner, mainly used to extract text and tables, easy to use Implemented the extraction of table data Half-frame table effect is poor Camelot Extracting Tabular Data from PDF Using Visual Methods Table extraction is better Mainly focused on extracting tabular data, with limited capabilities for handling non-tabular content Papermage Based on PDFPlumber, layout analysis is performed through deep model Provides graphical support and supports a variety of layout elements Applicable only for essay scenarios DeepDoc Provides table parsing, extraction, and recognition of images of various elements in PDF Rich identification element types, accurate simple table extraction Different elements are processed in isolation and their positions cannot be restored PDF-Extract-Kit Uses deep learning models to detect PDF layouts, providing formula detection, formula recognition, and OCR recognition Able to accurately recognize text and tables in PDF Image recognition may miss something, and text recognition is slow As can be seen from the table above, various tools have different advantages and disadvantages, and can extract text, tables, pictures and other contents from PDF documents. However, there is a key problem: the above open source tools all extract text, tables, pictures and other contents from PDF independently, without restoring the position, resulting in the elements of various modalities being separated, which cannot meet people's normal reading order. At the same time, this approach will cause the elements of various modalities to be unable to be contextually associated, affecting subsequent document slicing, and further affecting the retrieval and recall of the vector library.

[0006] Therefore, there is an urgent need for an intelligent parsing of PDF documents that can accurately extract text, tables, and images from PDF documents, while also realizing a method for restoring the positions of PDF document elements and constructing indexes of elements of different modalities. Summary of the invention

[0007] The present invention overcomes the deficiencies of the prior art and provides a method for document parsing and indexing construction that can accurately extract text, tables, and images from PDF documents and construct complete index elements.

[0008] The technical solution of the present invention is as follows: A document parsing and index building method specifically includes the following steps: 1) Layout detection step: Use the document layout analysis model to detect and classify the images, tables, titles, and text elements on the PDF document page; 2) Table image parsing and extraction step: extract and process the format and content of the tables and images existing in each page of the PDF obtained based on the layout detection; 3) Page recovery step: splice the elements according to the processed text, table, and image in the order of the page position, and remove invalid characters in the text; 4) Content segmentation step: Implement how to segment text, tables, and images to form a list of segmented paragraphs, and use it as the basis for vector library retrieval and recall; 5) Index building and storage steps: Create specific content that can directly match user questions and calculate similarity for the split paragraph list, and import it into the database.

[0009] Furthermore, the layout detection specifically includes the following steps: 1.1) Detection and classification steps: Use the layout detection model to obtain the attributes and element types of each page of the PDF document; the attributes include page number, page resolution, width and height; the element types include text, image, and table; the text includes title, plain text, image title, table title, and table comment; 1.2) Judgment step: judge the type of elements after detection and classification. If it is a table or image, directly save their location information in the PDF page; if it is text, use PyMuPDF to extract the text content in the specified rectangular box of a specific PDF page; The string redundancy elimination method is used to remove duplicates from the text, and then the text is sorted after removal of duplicates; 1.3) Saving step: Save the positions of the de-duplicated and sorted texts, tables and images on each page of the PDF obtained by the judgment processing.

[0010] Furthermore, in the process of image parsing and extraction of table images, after traversing each page of the PDF document, the image of the page will be re-identified and extracted, and the coordinates of the four vertices of the image will be saved. Secondly, the coordinates of the newly extracted image and the coordinate positions of the image extracted by the original layout detection model will be deduplicated. Finally, the original image will be cropped according to the image coordinates and uploaded to the public network storage system. The storage system returns the URL address of the image and splices the URL address of the image into a markdown representation.

[0011] Furthermore, the table processing in the table image parsing and extraction process includes simple tables and complex tables, both of which use HTML to represent and store tables, which can be correctly rendered into the markdown parser; Among them, a simple table refers to a table with a relatively regular structure and no cell merging; a complex table refers to a table with an irregular structure, multi-level cell merging, and no borders.

[0012] Furthermore, the content of the parsed PDF document is split according to The mark is split to get a list, and then the text content whose end character is not a period is merged backward to get a new list; Based on the latest list, further segment the element contents therein; the element contents include text, tables and images; For text, if the length of the text does not reach the specified length, continue to traverse the list to merge the text; if it reaches the specified length, store the previously merged text and jump to the next list element; For tables, retain the specified number of contexts before and after the table. If the number of rows in the table is too long, split the original table into multiple tables according to the number of rows in the table. For images, the specified amount of context before and after the image is also retained.

[0013] Furthermore, for plain text, the index is constructed and stored in the database using two methods: splitting by specific characters and the number of characters, and generating paragraph questions. Specifically splitting by specific characters and the number of characters sets the appropriate number of characters based on the text length supported by the vector model. Paragraph question generation uses LLM to summarize and extract questions that are more relevant to the paragraph for construction.

[0014] Furthermore, for a table during index building and storage, the context content of the table will be split, and then the cell row content of the table will be split; The context of the table includes the above description, the table title, the notes of the table below, and the text description below.

[0015] Furthermore, three index construction methods are adopted for table construction: table title, fine-grained index, and coarse-grained index.

[0016] Furthermore, during index construction and storage, the contextual content of the image will be split, including the image title and text description; and a multimodal large model will be used to summarize the image content and generate a summary description of the image.

[0017] Compared with the prior art, the present invention has the following advantages: The present invention realizes intelligent parsing for PDF documents, can accurately extract texts, tables and images in PDF documents, and simultaneously realizes a method for restoring the positions of PDF document elements and constructing indexes of elements of different modalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of traditional RAG document processing; Figure 2 It is a schematic diagram of the overall process framework of the present invention; Figure 3 This is an example diagram of layout detection of the present invention; Figure 4 An example diagram of the process of building an index for plain text; Figure 5 For the table case that needs to be parsed; Figure 6 For Figure 5 An example diagram of the process of index building for the table in the figure; Figure 7 For the image case that needs to be parsed; Figure 8 For Figure 7 An example diagram of the process of indexing images in the image; Fig. 9 This is an example table diagram after table extraction of the present invention; Fig.10 It is a part of the original pages of document 1 in the example document of the present invention; Fig.11 For Fig.10 The split result of the original page in ; Fig.12 It is a part of the original page of document 2 in the example document of the present invention; Fig.13 For Fig.12 The split result of the original page in ; Fig.14 It is a part of the original page of document 3 in the example document of the present invention; Fig.15 For Fig.14 The split result of the original page in ; Fig.16 It is a part of the original page of document 4 in the example document of the present invention; Fig.17 For Fig.16 The split result of the original page in ; Fig.18 It is a part of the original page of document 5 in the example document of the present invention; Fig.19 For Fig.18 The split result of the original page in ; Fig. 20 It is a part of the original page of document 6 in the example document of the present invention; Fig.21 For Fig. 20 The split result of the original page in ; Fig. 22 It is a part of the original page of document 7 in the example document of the present invention; Fig.23 For Fig. 22 The split result of the original page in ; Fig.24 An original page of a PDF document containing text and images in one embodiment of the present invention; Fig.25 For Fig.24 An example of the result obtained after splitting the original page. DETAILED DESCRIPTION

[0019] The embodiments of the present invention are described in detail below, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and cannot be used as a limitation of the present invention.

[0020] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the field to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0021] The numbers of the steps mentioned in each embodiment are only for the convenience of description and have no substantial sequence relationship. Different steps in each specific embodiment can be combined in different sequences to achieve the purpose of the invention.

[0022] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0023] like Figure 2 As shown, a document parsing and index building system includes layout detection, table image parsing and extraction, page recovery, content splitting and index building and storage. The document goes through the above five parts in sequence to build an index library of the document.

[0024] A document parsing and indexing component method specifically comprises the following steps: 1) Layout detection step: Use the document layout analysis model to detect and classify the areas of images, tables, titles, texts, etc. on the PDF document page. Layout detection specifically includes the following steps: 1.1) Detection and classification steps: Use the layout detection model to obtain the attributes and element types of each page of the PDF document. The attributes include page number, page resolution, width and height, and the element types include text, image, and table. The text includes title, plain text, image title, table title, and table comment.

[0025] 1.2) Judgment step: judge the type of elements after detection and classification. If it is a table or image, directly save their location information in the PDF page. If it is text, use PyMuPDF to extract the text content in the specified rectangular box of a specific PDF page. This step does not use the OCR text recognition model to recognize and extract text, which greatly improves the speed of text extraction. Because the layout detection model may have repeated detection or inaccurate text detection, such as Figure 3 As shown, the text in the light-colored frame and two dark-colored frames will be detected at the same time, as follows: Contents of light-colored boxes: 1. Reasons for topic selection and research value\n (1) Theoretical appeal: Artificial intelligence is participating in major changes in higher education.

[0026] Dark box 1 content: 1. Reasons for the topic selection and research value. Dark box 2 content: (1) Theoretical appeal: Artificial intelligence is participating in major changes in higher education.

[0027] At this time, the string redundancy elimination method will be used to deduplicate the text, and then sort it after deduplication.

[0028] 1.3) Saving step: Save the positions of the de-duplicated and sorted texts, tables and images on each page of the PDF obtained by the judgment processing.

[0029] 2) Table image parsing and extraction step: Extract and process the format and content of the tables and images existing in each page of the PDF based on the layout detection.

[0030] Specifically, because the layout detection model may miss images when identifying PDF document pages, the image processing will re-identify and extract the image of each page of the PDF document after traversing it, and save the coordinates of the four vertices of the image. Secondly, the newly extracted image coordinates and the coordinate positions of the image extracted by the original layout detection model are deduplicated. Finally, the original image is cropped according to the image coordinates and uploaded to the public network storage system. The storage system returns the URL address of the image and splices the URL address of the image into a markdown representation.

[0031] Table processing includes simple tables and complex tables. There are various ways to parse tables, such as markdown, html, latex, etc., but this solution uses html to represent and store tables. This representation method can better preserve the original table structure and can be correctly rendered into the markdown parser.

[0032] A simple table refers to a table with a relatively regular structure and no cell merging. A complex table refers to a table with an irregular structure, multi-level cell merging, and no borders.

[0033] 3) Page recovery step: refers to the splicing of elements according to the position order of the page based on the processed text, table, and image, and the removal of invalid characters in the text, such as extra spaces, special marks, etc., to finally obtain various elements with the original position order. The specific page recovery is to rearrange the position order of the page corresponding to the markdown content of the parsed image, the html content of the table, the plain text content, and the various elements according to the parsed markdown content of the image, the html content of the table, the plain text content, and the various elements to obtain the final parsed content.

[0034] 4) Content splitting step: The core is how to better split text, tables, and images. The quality of content splitting is the basis of vector library retrieval recall. Most tools and research focus more on content parsing, while ignoring the splitting of multimodal elements after parsing. Because the parsed PDF page element content does not have paragraph information, it is necessary to merge the content. The specific processing of this solution is as follows: In the content splitting, the content of the parsed PDF document is divided into The mark is split to get a list, and then the text content whose end character is not a period is merged backward to get a new list. The purpose of this is to retain the information of the original paragraph as much as possible.

[0035] Based on the latest list, since too large a block will result in too long a context when LLM is combined with the knowledge obtained by retrieval, affecting the final reply effect, it is necessary to further segment the element content. The element content is divided into three types, including text, table and image.

[0036] For text, if the length of the text does not reach the specified length, continue to traverse the list to merge the text. If it reaches the specified length, store the previously merged text and jump to the next list element.

[0037] For a table, retain the specified number of contexts before and after the table. If the number of rows in the table is too long, split the original table into multiple tables based on the number of rows in the table.

[0038] For images, the specified amount of context before and after the image is also retained.

[0039] After the above splitting steps, you will get a list of split chunk paragraphs (including text, tables, and images).

[0040] 5) The index building and storage step is to create specific content that can directly match the user's questions and calculate the similarity based on the split chunk paragraph list, and then import it into the database.

[0041] Specifically, for plain text, we use two methods to build the index: splitting by specific characters and the number of characters, and generating paragraph questions. Among them, splitting by specific characters and the number of characters is to set the appropriate number of characters according to the text length supported by the vector model. Paragraph question generation is to use LLM to summarize and extract questions that are more relevant to the paragraph from the paragraph for construction. The purpose of this is to better match the user's questions and improve the hit rate of recalled knowledge. Specifically, a piece of plain text is used as an example. The content of the plain text example is as follows: Chapter 1 Introduction 1. Reasons for the topic selection and research value The new generation of information technology represented by artificial intelligence is becoming a key factor in leading scientific and technological change and industrial revolution. Artificial intelligence has a strong leading goose effect. It is not only a potential driving force for economic and social development, but also a subversive force for promoting social change and progress. It presents the characteristics of informatization, networking, digitization, and intelligence, which are parallel, related, and progressive, and together constitute the social development trend in the digital era. From an economic perspective, higher education is a public product or quasi-public product. [1] , is a social activity that cultivates high-quality professional talents and serves economic and social development in the digital era. Compared with different stages and levels of education, the economic attributes of higher education determine its close connection with the digital transformation of society. The two are born in the same direction and promote each other. In the digital era, thinking about the innovation and transformation of higher education driven by artificial intelligence, it is urgent to study the deep relationship between society, technology and education with the overall thinking of social development. This is the basic starting point for studying future education and the educational mission based on the economic and social development in the new era.

[0042] 1. Theoretical Appeal: Artificial Intelligence is Participating in Major Changes in Higher Education 1. Artificial intelligence is an inevitable trend driving changes in higher education.

[0043] The leapfrog development of society is accompanied by technological revolution. The three industrial revolutions recognized in human history have had a huge impact on the development of education. The industrial revolution indirectly promoted educational reform by transforming social productivity and production relations, especially in the field of higher education. The functions of higher education have changed, and the discussion on the nature of higher education has seen the emergence of superstructure theory and productivity theory. As far as the current status of education is concerned, it is still based on batch, standardized, and centralized talent training based on the classroom teaching system, following the traditional education model. Artificial intelligence is considered by some scholars to be a transformative technology that promotes the fourth industrial revolution in human society. Based on this, the academic community has also had great expectations for artificial intelligence to drive higher education reform, setting off a wave of research on artificial intelligence education. ".

[0044] Figure 4The results of text splitting and paragraph question generation for the above plain text are shown in Figure 2, which shows a good match between questions and index construction.

[0045] For a table, the context of the table is split, and then the cell row content of the table is split. The context of the table includes the previous description, the table title, the table notes below, the text description below, etc. For the table, three index construction methods are used: table title, fine-grained index, and coarse-grained index. Figure 5 and Figure 6 As shown, in order to adapt to various user questions about specific content, some content, and all content of the table, three construction methods are adopted here: table title, fine-grained index, and coarse-grained index. This rich table index construction method can better match the semantics of user questions.

[0046] For images, the context of the image will be split, including the title and text description of the image. Considering the effectiveness of image content information, only using OCR to recognize image content cannot fully express the image, so a multimodal large model is used to summarize the image content and generate a summary description of the image. The specific image index construction example is as follows: Figure 7 and Figure 8 shown.

[0047] Finally, the index of the organized text, table, and image is vectorized. This vector model can represent the index content with both dense vectors and sparse vectors. The dense vector focuses on the deep semantic meaning of the content, while the sparse vector focuses on the importance of keywords in the content. Using two vector representations can match and recall from different dimensions.

[0048] In summary, three different types of complex PDF documents are used to verify this solution. The three different types include PDF documents with text + image, text + table, and text + table + image. Specifically, 7 documents were selected, such as Figures 10 to 23 As shown, the details are as follows: Document 1 is a manual for the operation process of a company applying for a project. Some pages of the manual contain two or more images and text content. The specific partial extraction results are as follows: Fig.10 and Fig.11 As shown, the two images appearing in the page and the plain text content contained in the page can be accurately extracted according to the order of the page.

[0049] Document 2 is a manual on the operation process of tax declaration, in which some pages have two or more images, image titles, and context content appearing at the same time. The specific partial extraction results are as follows: Fig.12 and Fig.13As shown, the two images appearing in the page and the plain text content contained in the page can be accurately extracted according to the order of the page.

[0050] Document 3 is a list of anticancer drugs and rare disease drugs. In the document, table cells are merged horizontally, and the content of the table cells is drugs. The text content is relatively complex, such as fluzoparib, dalsilil isethionate, mitoxantrone hydrochloride, etc. The specific partial extraction results are as follows: Fig.14 and Fig.15 shown.

[0051] Document 4 is a corporate income tax return form. In the document, there are cases where table cells are merged horizontally and vertically, and the table text content is not standardized. The table format and content are extremely complex. In this case, the table extraction results can basically meet the requirements and only require slight manual modification later. The specific partial extraction results are as follows: Fig.16 and Fig.17 shown.

[0052] Document 5 is a list of duty-free shops for foreign exchange commodities. In the document, there are cases where the table cell contents are merged vertically and the table text contents are in multiple formats (Chinese, numbers, letters), etc. For example: 1st Floor, Building 2\n, Qingdao Zhuoyueli Shopping Mall, No. 31, Longcheng Road, Shibei District, Qingdao, S2\n Shangshu, Shangyue West Plaza, No. 12-2, Jimo Road, Pudong New District, Shanghai, etc. In this case, the table and its contents can also be accurately extracted. The specific partial extraction results are as follows Fig.18 and Fig.19 shown.

[0053] Document 6 is an annotated table of national economic industry classification. The table in the document has obvious borders and there are empty characters in the cell content. In this case, empty cells and non-empty cells can be accurately extracted. The specific partial extraction results are as follows: Fig. 20 and Fig.21 shown.

[0054] Document 7 is a tax incentive policy. The document contains pages with images as the background, as well as tables and text. The specific partial extraction results are as follows: Fig. 22 and Fig.23 shown.

[0055] The effectiveness of the solution is tested from the dimensions of table extraction accuracy, table content extraction accuracy, image extraction accuracy, layout recovery, and speed. The specific verification results are shown in Table 2 below: Table 2 Complex Types time consuming Pages Total number of tables Table extraction accuracy Table content extraction accuracy Total number of images Image extraction accuracy Document 1 3min9s 39 \ \ \ 26 100% Document 2 2min42s 100 \ \ \ 211 100% Document 3 3.6s 2 2 100% 100% \ \ Document 4 18min48s 155 68 100% 86% \ \ Document 5 2s 2 2 100% 100% \ \ Document 6 25min36s 406 406 100% 97.8% \ \ Document 7 26min 402 30 100% 90% 2 100% As can be seen from Table 2, this solution has a high accuracy in extracting PDF document images and tables. When extracting table content, the structure of the table and the content of a single cell table will be manually compared at the same time, and ultimately good results can be achieved in extracting table content. Because tables come in a variety of forms, such as complete borders and no merged cells, merged cells, incomplete borders, etc., the accuracy of table content extraction will be affected. Through analysis, it is found that the accuracy of table content extraction will be higher when the table borders are complete. In terms of time consumption, since the length of text content contained in different documents is different, if the text length is too long, the main time consumption is concentrated on the construction of the problem index.

[0056] Specifically, a PDF document titled "Research on Artificial Intelligence Driving Higher Education Transformation in the Digital Age" was used, which contains three types of elements: plain text, tables, and images. Figure 5 The following is an original table in the document, titled "Table 1.1 Foreign AI Education Policies (Part)". After extracting the table, we can get Fig. 9 From the results, it can be seen that while retaining the original table structure, this scheme can accurately extract the content in the table cells.

[0057] Fig.24 As shown, the original page is a pdf document page containing text and images, which is obtained after splitting Fig.25 As can be seen from the figure, while ensuring the order of the image and text, this solution can correctly extract the context text content of the image and the title of the first-level image.

[0058] In summary, through analysis and verification, this solution can realize the recovery of the page content (text, tables, pictures) of PDF documents, and this solution makes up for the disadvantages of only recognizing and parsing PDF documents through deep learning models, and has good recognition and extraction accuracy. This solution also provides a method for building indexes for different types of content. This refined PDF document processing method can retain the location and display order of the original document, and at the same time lays a good foundation for the recall of the vector library.

[0059] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the concept of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A document parsing and indexing method, characterized in that: The specific steps include: 1) Layout detection step: Use the document layout analysis model to detect and classify the images, tables, titles, and text elements on the PDF document page; 2) Table image parsing and extraction step: the format and content of the tables and images existing in each page of the PDF obtained based on the layout detection are extracted and processed; 3) Page recovery step: refers to the splicing of elements according to the processed text, table, and image in the order of the page position, and at the same time, the invalid characters in the text will be removed; 4) Content segmentation step: This is to implement how to segment text, tables, and images, form a list of segmented paragraphs, and use it as the basis for vector library retrieval and recall; 5) Index building and storage steps: Create specific content that can directly match user questions and calculate similarity based on the split paragraph list, and import it into the database.

2. A document parsing and indexing method according to claim 1, characterized in that: Layout detection specifically includes the following steps: 1.1) Detection and classification steps: Use the layout detection model to obtain the attributes and element types of each page of the PDF document; the attributes include page number, page resolution, width and height; the element types include text, image, and table; the text includes title, plain text, image title, table title, and table comment; 1.2) Judgment step: judge the type of elements after detection and classification. If it is a table or image, directly save their location information in the PDF page; if it is text, use PyMuPDF to extract the text content in the specified rectangular box of a specific PDF page; The string redundancy elimination method is used to remove duplicates from the text, and then the text is sorted after removal of duplicates; 1.3) Saving step: Save the positions of the de-duplicated and sorted texts, tables and images on each page of the PDF obtained by the judgment processing.

3. A document parsing and indexing method according to claim 1, characterized in that: Image processing during table image parsing and extraction, after traversing each page of the PDF document, the image of the page will be re-identified and extracted, and the coordinates of the four vertices of the image will be saved. Secondly, the coordinates of the newly extracted image and the coordinate positions of the image extracted by the original layout detection model will be deduplicated. Finally, the original image will be cropped according to the image coordinates and uploaded to the public network storage system. The storage system returns the URL address of the image and splices the URL address of the image into a markdown representation.

4. A document parsing and indexing method according to claim 1, characterized in that: The table processing in the table image parsing and extraction process includes simple tables and complex tables, both of which use HTML to represent and store tables, which can be correctly rendered into the markdown parser; Among them, a simple table refers to a table with a relatively regular structure and no cell merging; a complex table refers to a table with an irregular structure, multi-level cell merging, and no borders.

5. A document parsing and indexing method according to claim 1, characterized in that: In the content splitting, the content of the parsed PDF document is divided into The mark is split to get a list, and then the text content whose end character is not a period is merged backward to get a new list; Based on the latest list, further segment the element contents therein; the element contents include text, tables and images; For text, if the length of the text does not reach the specified length, continue to traverse the list to merge the text; if it reaches the specified length, store the previously merged text and jump to the next list element; For tables, retain the specified number of contexts before and after the table. If the number of rows in the table is too long, split the original table into multiple tables according to the number of rows in the table. For images, the specified amount of context before and after the image is also retained.

6. A document parsing and indexing method according to claim 1, characterized in that: For plain text, the index is constructed by two methods: splitting by specific characters and the number of characters, and generating paragraph questions. The specific character and character number splitting is to set the appropriate number of characters according to the text length supported by the vector model. The paragraph question generation is to use LLM to summarize and extract the questions that are more relevant to the paragraph from the paragraph for construction.

7. A document parsing and indexing method according to claim 1, characterized in that: During index building and storage, for a table, the context content of the table will be split, and then the cell row content of the table will be split; The context of the table includes the above description, the table title, the notes of the table below, and the text description below.

8. A document parsing and indexing method according to claim 7, characterized in that: For tables, three index construction methods are used: table title, fine-grained index, and coarse-grained index.

9. A document parsing and indexing method according to claim 1, characterized in that: During index construction and storage, the context of images will be split, including the image title and text description. A multimodal large model will be used to summarize the image content and generate a summary description of the image.

Images