Document analysis method and system, electronic equipment, storage medium and program product

Abstract

The invention discloses a document analysis method and system, electronic equipment, a storage medium and a program product, and belongs to the technical field of natural language processing and document analysis. Comprising the steps that original OpenXML data of a Word document to be analyzed is read, each top-level label in the original OpenXML data is recognized, each recognized top-level label serves as a data block, and the data type of the data block is determined. And under the condition that the data type of one data block is a target data type which cannot be analyzed by a Word document analysis system, constructing a first prompt according to the content of the data block, and inputting the first prompt into the first large language model to obtain an analysis result of the data block. And splicing an analysis result of the first large language model for the data block of which the data type is the target data type and an analysis result of the Word document analysis system for the data block of which the data type is not the target data type to obtain an analysis result of the to-be-analyzed Word document. The technical effect of improving the accuracy and integrity of document analysis can be achieved.

Description

Technical Field

[0001] This application belongs to the field of natural language processing and document parsing technology, specifically relating to a document parsing method, system, electronic device, storage medium, and program product. Background Technology

[0002] In Retrieval-Augmented Generation Knowledge Base (RAG) applications, document parsing technology is the core foundation for achieving high-quality information retrieval and content generation. Currently, knowledge base systems rely on the accurate parsing of office documents to construct structured data sources. Among these, Word and Excel documents are among the most commonly used document formats, and their parsing quality directly affects the reliability of subsequent retrieval and generation processes.

[0003] Existing document parsing technologies mainly use dedicated software libraries to directly read various office documents such as Word / .docx, Excel / .xlsx, extract text, tables, charts and other elements, and convert them into programmable data structures. Alternatively, for scanned documents or documents containing images, optical character recognition technology is used for text conversion to extract structured and unstructured data.

[0004] However, in Word document parsing, mainstream software libraries can only process basic text content and lack the ability to effectively extract complex structures such as mathematical formulas; while Excel documents have very low rule base support due to problems such as complex headers, irregularly merged cells, single-page blocks, and extremely long tables, making it difficult to cope with the complex table parsing needs common in real-world scenarios, which seriously restricts the construction and application of the RAG knowledge base. Summary of the Invention

[0005] The purpose of this application is to provide a document parsing method, system, electronic device, storage medium, and program product that can solve the problems of low parsing accuracy and poor applicability of existing document parsing technologies when processing complex format documents.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a document parsing method, including: Read the raw OpenXML data of the Word document to be parsed; Identify each top-level tag in the original OpenXML data, treat each identified top-level tag as a data block, and determine the data type of the data block; When a data block has a data type that the Word document parsing system cannot parse, a first prompt is constructed based on the content of the data block and input into the first large language model to obtain the parsing result of the data block. The parsing result of the first language model for a data block whose data type is the target data type is concatenated with the parsing result of the Word document parsing system for a data block whose data type is not the target data type to obtain the parsing result of the Word document to be parsed.

[0007] In some embodiments, the target data type includes at least mathematical formulas; The process of constructing a first prompt based on the content of the data block and inputting it into the first large language model to obtain the parsing result of the data block includes: Based on the preset first prompt template and the content of the data block, the first prompt is constructed, and the content of the data block is converted into a standard format mathematical expression through the first large language model.

[0008] In some embodiments, the target data type further includes at least a table; The step of constructing a first prompt based on the content of the data block and inputting it into the first large language model to obtain the parsing result of the data block includes: Based on the preset second prompt template and the content of the data block, a first sub-prompt is constructed and input into the first large language model to obtain the table area range and cell content of the data block; Based on the preset third prompt template and the content of each cell of the data block, a second sub-prompt is constructed and input into the first large language model to obtain the original header content of the data block. According to the preset header compression rules, the original header content of the data block is compressed into a single-row header; Based at least the fourth prompt template and the single-row header, a third sub-prompt is constructed and input into the first large language model to obtain the primary key column of the data block; The data columns of the data block are determined based on the primary key column and the table range of the data block; If the number of columns in the data block does not exceed a preset column number threshold, extract the header field of each row and the cell content of that row in the data column, and obtain the parsing result of the data block based on the primary key column of the data block.

[0009] In some embodiments, compressing the original header content of the data block into a single-row header according to a preset header compression rule includes: If the original header content contains merged cells, identify the expansion type of the merged cells; When the expansion type of the merged cell is expanded by row, the merged cell is expanded by row, and the content of the first row of merged cells is retained; When the expansion type of the merged cell is expanded by column, the merged cell is expanded by column, and the contents of the merged cell are copied to each column; For each column of multi-row headers, the contents of each expanded row of cells are concatenated in a top-to-bottom order to obtain the single-row header.

[0010] In some embodiments, after determining the data columns of the data block, the method further includes: If the number of columns in the data block exceeds the preset column number threshold, the data block is divided into multiple sub-data column groups according to the preset specified number of columns. Based on the principle of retaining the primary key column, each sub-data column group is combined with the primary key column to generate various sub-tables; Iterate through each sub-table, extract the header fields of each row and the cell content of that row in the data column, and obtain the standardized text of each sub-table based on the primary key column of each sub-table to generate the parsing result of the data block.

[0011] Secondly, embodiments of this application provide a document parsing method, including: Read the raw OpenXML data of the Excel document to be parsed; For each sheet of the Excel document to be parsed, a third prompt is constructed based on the original OpenXML data and the preset fifth prompt template, and then input into the first large language model to obtain the table area range and cell content of each independent table. For each independent table, a fourth prompt is constructed based on the preset sixth prompt template and input into the first large language model to obtain the original table header content of each independent table; According to the preset header compression rules, the original header content of each independent table is compressed into a single-row header; Based at least on the preset seventh prompt template and the single-row header of each independent table, a fifth prompt is constructed and input into the first large language model to obtain the primary key column of each independent table; The data columns of each independent table are determined based on the primary key column and the table range of each independent table; If the number of columns in the data columns of each independent table does not exceed the preset column number threshold, extract the header field of each row and the cell content of the row in the data column, and output each independent table according to the primary key column of each independent table to obtain the parsing result of the Excel document to be parsed.

[0012] Thirdly, embodiments of this application provide a document parsing system, including: A Word document parsing system is used to parse data blocks whose data type is not the target data type, so as to obtain the parsing result of the data blocks whose data type is not the target data type; The first major language model is used to implement the document parsing method as described in the first aspect, or to implement the document parsing method as described in the second aspect; A document library for storing the parsing results of a Word document to be parsed obtained according to the document parsing method as described in the first aspect, and / or the parsing results of an Excel document to be parsed obtained according to the document parsing method as described in the second aspect; The second major language model is used to search for M document slices in the document library based on the search terms input by the user, and to select at least the first m document slices based on semantic similarity for information extraction and integration, so as to obtain the result matching the search terms and reply to the user, where 1≤M, 1≤m≤M.

[0013] Fourthly, embodiments of this application provide an electronic device, including: processor; Memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the document parsing method as described in the first aspect, or to implement the document parsing method as described in the second aspect.

[0014] Fifthly, embodiments of this application provide a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a terminal's processor, enables the terminal to execute the document parsing method as described in the first aspect, or to implement the document parsing method as described in the second aspect.

[0015] Sixthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the document parsing method as described in the first aspect, or implements the document parsing method as described in the second aspect.

[0016] In this embodiment, by introducing a large language model to handle special data types that the Word document parsing system cannot parse, and combining the output results of the traditional parsing system, the parsing problem of complex structures such as mathematical formulas is effectively solved. It has the advantages of being able to effectively parse special data types such as mathematical formulas in Word documents, avoiding information loss, and improving the accuracy and completeness of document parsing. Attached Figure Description

[0017] Figure 1 A flowchart illustrating the steps of a document parsing method provided in an embodiment of this application is shown. Figure 2 This illustration shows a flowchart of the document parsing method provided in an embodiment of this application; Figure 3 A flowchart of a document parsing method provided in another embodiment of this application is shown; Figure 4 An exemplary flowchart of a document parsing method provided in an embodiment of this application is shown; Figure 5 A schematic diagram of the structure of a document parsing system provided in an embodiment of this application is shown; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] The document parsing method, system, electronic device, storage medium, and program product provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0021] This patent relates to the field of artificial intelligence technology, specifically to a knowledge base system based on Retrieval-Augmented Generation (RAG) and its key document parsing technologies. It is particularly suitable for improving the information extraction and processing capabilities of office documents (such as Word and Excel format files) to optimize the quality of knowledge base question answering.

[0022] The core advantage of the RAG knowledge base system lies in its ability to integrate information retrieval and generative models. Its generated content combines coherence with timeliness and accuracy, significantly improving the quality of answers to complex questions. The typical result generation process of this system includes the following key steps: Query rewriting: The semantics of the search terms entered by the user are optimized and rewritten to align with the representation of the document space to be searched, thereby improving the accuracy of subsequent searches; Query and retrieval processing: Based on technologies such as vector retrieval and full-text keyword retrieval, locate the most relevant document slices in the structured document library; Document slice reordering: The retrieved document slices are prioritized and selected as the Top K most relevant slices by using methods such as semantic similarity calculation and ranking models. Results generation and processing: Utilizing the semantic understanding and information integration capabilities of the large model, the selected document slices are refined and integrated into the final answer.

[0023] The effectiveness of the query and retrieval step directly determines the overall quality of the answer; and the core of this step relies on document parsing technology, especially the ability to extract structured and unstructured information from mainstream office documents such as Word (.docx) and Excel (.xlsx). This type of technology aims to efficiently and accurately parse diverse data in office documents to support scenarios such as automated workflows, data analysis, and knowledge base construction.

[0024] Currently, file parsing technology mainly employs the following two methods: (1) Parsing based on a dedicated library: The document is directly read and parsed through a predefined software library, which can identify basic elements such as text, tables, charts, and formulas, and convert them into data structures that can be operated by the program. However, there are significant limitations: the library lacks effective parsing capabilities for complex structures such as mathematical formulas in Word documents; for Excel documents, the existing library has extremely low parsing support for high-frequency scenarios such as complex table headers (such as multi-level nested table headers), irregularly merged cells, block tables within a single page, and ultra-long cross-page tables, which is difficult to meet the requirements.

[0025] (2) Optical Character Recognition (OCR): This method is only used to process scanned or image-based text content in Word documents, enabling editability through image-to-text conversion. However, due to limitations such as image quality, character distortion, and interference noise, the OCR recognition error rate is relatively high; moreover, this method has limited application scenarios in Excel documents and cannot solve the problem of table structure parsing.

[0026] In summary, existing manual rule-based parsing schemes are ill-equipped to handle the complex structural characteristics of office documents, resulting in insufficient completeness and accuracy of document information extraction in the RAG system, which hinders the effective improvement of the quality of knowledge base question answering.

[0027] For example, when parsing Word documents containing academic papers with a large number of mathematical formulas, traditional parsing systems can only recognize... <w:p>and <w:tbl>The system can only parse regular text content within tags, but it cannot effectively parse mathematical expression regions embedded in the document flow. When the system encounters... <m:math>When parsing OpenXML formula data wrapped in tags, due to a lack of understanding of the hierarchical relationships of mathematical symbols, the data block is either skipped or garbled content is output, resulting in the permanent loss of critical formula information. The resulting parsing results show the formula portion as blank spaces or meaningless character sequences, breaking the semantic chain of the document. This severely affects the accurate representation of professional domain content by the knowledge base, ultimately reducing the credibility and usability of the entire knowledge base system.

[0028] Figure 1 A flowchart illustrating the steps of the document parsing method provided in this application by way of an embodiment is shown. (Refer to...) Figure 1 This application provides a document parsing method, including: S10. Read the raw OpenXML data of the Word document to be parsed.

[0029] Specifically, reading the raw OpenXML data of a Word document to be parsed refers to the process of directly obtaining the underlying XML structure representation of the document. This can be achieved by using standard document processing libraries to parse the XML stream or by accessing a specific XML file within a .docx compressed package via the file system. For example, the Apache POI library can be used to read the content of word / document.xml, or the python-docx tool can be used to extract the raw data stream of the Word document to be parsed. By reading the raw OpenXML data of the Word document to be parsed, the basic input for the parsing process is obtained, avoiding errors that may be introduced by relying on rendering layer data.

[0030] S20. Identify each top-level tag in the original OpenXML data, treat each identified top-level tag as a data block, and determine the data type of the data block.

[0031] Specifically, identifying each top-level tag in the raw OpenXML data refers to the operation of locating the main structural units of the document. This can be achieved by using an XML parser to traverse the DOM tree to extract the root node or by matching tag boundaries using regular expressions.

[0032] For example, top-level tags include at least <w:p>Text paragraph tags, <w:tbl>Table labels and <m:math>Mathematical formula labels, through recognition <w:p>Text paragraph tags <w:tbl>Table labels and <m:math>Mathematical formula labels identify data blocks. The data type of each data block is determined by the label name or attributes, for example... <w:p>The tag corresponds to the text paragraph type. <m:math>The tag corresponds to the mathematical formula type. <w:tbl>The label corresponds to the table type.

[0033] In this embodiment, each top-level tag in the original OpenXML data is identified and treated as an independent data block. The data type of each data block is determined by identifying the top-level tag type, so as to distinguish whether it is a target data type that the Word document parsing system cannot parse, thus ensuring the fine-grained splitting of the Word document to be parsed.

[0034] S30. If the data type of a data block is a target data type that the Word document parsing system cannot parse, construct a first prompt based on the content of the data block and input it into the first large language model to obtain the parsing result of the data block.

[0035] Specifically, the parsing result of a data block refers to the structured data or standardized information format output after parsing by the first major language model, which can be used in subsequent processes. Target data types that the Word document parsing system cannot parse can be identified using a predefined list of unsupported types. For example, when the predefined list of unsupported types is... <m:math>Tags and <w:tbl>When using table tags, if the data block's data type is a mathematical formula or a table, construct the first prompt and input it into the first large language model to obtain the parsing result for that data block.

[0036] For example, for data blocks of text paragraph type, the parsing result means that the basic structure of the paragraph is preserved and it is a continuous string or a text sequence separated by paragraphs; for data blocks of mathematical formula type, the parsing result means that the LaTeX standardized mathematical expression output by the first language model after processing the OpenXML formula data, with additional formatting constraints such as tag wrapping and Chinese variables being handled with \text{}; for data blocks of table type, the parsing result means that the corresponding structured data, such as two-dimensional arrays, HTML table code, or JSON objects.

[0037] In constructing the first prompt, this can be achieved by embedding the content of data blocks into a natural language description template. For example, for... <m:math>Mathematical formula tags, when the natural language description template is: "Please convert the following OpenXML format formulas to LaTeX format. Requirements: 1. Add the following tags at the beginning and end..." <latex>< / latex> 1. Mark fields; 2. Enclose Chinese variables in \text{}; 3. Do not include any extra content in the output besides the LaTeX content; 4. If the input is empty, output an empty character, and the output should not deviate from the input content. When the OpenXML content is: ", by <m:math>The content of the data block corresponding to the mathematical formula label is embedded at the end of the natural language description template and input into the first language model to obtain the parsing result of the data block.

[0038] S40. The parsing results of the first language model for data blocks whose data type is the target data type are concatenated with the parsing results of the Word document parsing system for data blocks whose data type is not the target data type, to obtain the parsing results of the Word document to be parsed.

[0039] Specifically, the concatenation method is to merge text fragments according to the original order of the data blocks. For example, the parsing results are concatenated into continuous text according to the structure index of the Word document to be parsed, so as to obtain the parsing results of the Word document to be parsed.

[0040] For example, Figure 2 A flowchart illustrating the steps of the document parsing method provided in this application by way of an embodiment is shown. (Refer to...) Figure 2 After reading the raw OpenXML data of the Word document to be parsed, it identifies... <w:p>< / w:p> Tags are used to distinguish a text paragraph by recognizing <w:tbl>< / w:tbl> Labels are used to distinguish a table, by recognition <m:math> <m:math>Tags are used to distinguish a mathematical formula, and the data type of each data block is determined by identifying each top-level tag type.

[0041] After distinguishing the data type of each data block, the Word document parsing system can only process... <w:p>< / w:p> Text paragraph, unable to be parsed <m:math>Mathematical formulas and <w:tbl>< / w:tbl> Table. Therefore, for <m:math>Mathematical formulas and <w:tbl>< / w:tbl> The table needs to be converted into a larger model for analysis.

[0042] In other words, in this example, for known identifiable data types (such as...) <w:p>< / w:p> (Text paragraphs) can be extracted using traditional parsing methods such as Word document parsing systems. <w:p>< / w:p> The content within a text paragraph is output as text; for unrecognized data types (such as...), <m:math>Mathematical formulas and <w:tbl>< / w:tbl> The data (tables) needs to be recognized and output as text using a large model (i.e., the first large language model). After all data blocks have been processed, a traditional parsing scheme is used to merge the data blocks in their original order and output them as text.

[0043] In this embodiment, data types are distinguished by top-level tags, and different parsing mechanisms are invoked accordingly. A traditional parsing system is used to process data blocks of non-target data types to maintain basic efficiency, while the first major language model is used to process data blocks of target data types to cover special content. This achieves complete extraction and structured output of the information from the Word document to be parsed. It solves the problem of incomplete information extraction under complex data types in traditional solutions and ensures the reliability of the parsing process.

[0044] In some embodiments, the target data type includes at least mathematical formulas.

[0045] S30. Based on the content of this data block, construct the first prompt and input it into the first large language model to obtain the parsing results of this data block, including: S31A. Based on the preset first prompt template and the content of the data block, construct the first prompt, and convert the content of the data block into a standard format mathematical expression through the first large language model.

[0046] Specifically, the target data type, including at least mathematical formulas, refers to specific data types that traditional Word document parsing systems cannot handle during document parsing. The preset first prompt template can be a text framework containing instruction constraints and formatting specifications, designed to provide structured input guidance for the first language model and avoid interference from irrelevant information. Constructing the first prompt specifically refers to embedding the actual OpenXML content of the data block into a specified location within the preset first prompt template. Its purpose is to generate targeted input for the first language model, ensuring that the parsing task focuses on the core content of the current data block. The first language model can be a natural language processing model based on deep learning, capable of processing unstructured data using semantic understanding and generation capabilities. Standardized mathematical expressions can be standardized markup languages ​​such as LaTeX, used to achieve precise expression of mathematical semantics and compatibility with subsequent systems.

[0047] For example, when the data type of the data block is identified as a mathematical formula, according to the preset first prompt template: "Please convert the following OpenXML format formulas to LaTeX format. Requirements: 1. Add the following to the beginning and end..." <latex>< / latex> 1. Mark fields; 2. Enclose Chinese variables in \text{}; 3. Do not include any extra content in the output besides the LaTeX content; 4. If the input is empty, output an empty character, and the output should not deviate from the input content. The OpenXML content is: ", append the actual OpenXML content of this data block to the end of the template to construct the complete first prompt. This first prompt is input into the first large language model, which outputs a standard LaTeX format mathematical expression, such as converting a complex OpenXML formula into "\int_{0}^{\infty} e^{-x^2} dx = \frac{\sqrt{\pi}}{2}". This standard format mathematical expression is then integrated into the parsing results of the Word document to be parsed.

[0048] This application's embodiments address the blind spots in traditional parsing systems' handling of mathematical formulas by incorporating mathematical formulas into the target data type category and designing a prompt construction mechanism that integrates a preset first prompt template with the data block content. Specifically, when the data block's data type is identified as a mathematical formula, the system constructs a first prompt based on the preset first prompt template combined with the actual content of the data block. This process effectively filters out interference from non-semantic elements such as fonts and font sizes in the original OpenXML data, allowing the first language model to focus on formula structure parsing. Subsequently, the first language model performs a transformation operation, mapping the lengthy OpenXML formula data into a compact, standard-format mathematical expression. This mapping process not only compresses the data volume but also achieves standardized expression of formula semantics. The final output can be directly embedded into the parsing results of the Word document to be parsed. Through the dynamic combination of the first prompt template and the data block content, the integrity of the original formula information is preserved, and the standardized output meets the needs of subsequent parsing result splicing and knowledge base applications. It effectively solves the problem that traditional document parsing systems cannot accurately handle complex structures such as mathematical formulas, realizes the semantic conversion of OpenXML format mathematical formulas, ensures the integrity and usability of document parsing results, and provides structured mathematical expression support for RAG knowledge base applications.

[0049] In some embodiments, the target data type may include at least a table.

[0050] S30. Based on the content of this data block, construct the first prompt and input it into the first large language model to obtain the parsing results of this data block, including: S31B. Based on the preset second prompt template and the content of the data block, construct the first sub-prompt and input it into the first large language model to obtain the table area range and the content of each cell of the data block.

[0051] Specifically, the target data type includes at least tables, which means incorporating the table structure into the first language model's auxiliary parsing scope and using a classification mechanism based on OpenXML tag recognition to identify the target data type; the second prompt template refers to the instruction framework for identifying the physical structure of the table, implemented using a text template containing instructions for extracting region boundaries, which is used to guide the large language model to accurately locate the spatial range of the table.

[0052] For example, before constructing the first prompt, the first N rows of data in the tables within the data block are converted to Markdown format. The default second prompt template is: "Please identify all the independent tables from the following table content. For each table, output A = top-left row number; B = top-left column number; C = total number of rows; D = total number of columns. Note that only these contents should be answered, and each independent table should be formatted using..." <data> A, B, C, D< / data> Wrap the tags. The table's Markdown format content is: "By embedding the first N rows of Markdown format data into the end of the preset second prompt template, the table area range (i.e., the number of rows at the top left, the number of columns at the top left, the total number of rows, and the total number of columns) and cell content are obtained, so as to facilitate the extraction of the original table header content of the data block.

[0053] S32B: Based on the preset third prompt template and the content of each cell in the data block, construct the second sub-prompt and input it into the first large language model to obtain the original header content of the data block.

[0054] For example, the preset third prompt template could be: "Please determine the first X rows as headers from the following table content, and the data starting from row X+1. Note that you should only answer the first X rows, and use..." <data>X< / data> Wrap the data in a tag. The table's Markdown format content is: . "By embedding the content of each cell of this data block into the end of a preset third prompt template, the original header content of the data block is obtained."

[0055] S33B. According to the preset header compression rules, compress the original header content of the data block into a single-row header.

[0056] Specifically, the header compression rule refers to the processing logic of converting multi-row headers into single-row headers. This can be achieved through generalized methods such as semantic merging or content concatenation, in order to eliminate the obstacles to data mapping caused by differences in header hierarchy.

[0057] In some embodiments, S33B, according to a preset header compression rule, compresses the original header content of the data block into a single-row header, including: S33B.1 When there are merged cells in the original header content, identify the expansion type of the merged cells.

[0058] S33B.2 When the merged cell expansion type is set to expand by row, expand the merged cells by row and retain the content of the first row of merged cells.

[0059] S33B.3 When the expansion type of merged cells is set to expand by column, expand the merged cells by column and copy the contents of the merged cells to each column.

[0060] S33B.4 For each column of multi-row headers, concatenate the contents of each expanded row of cells in a top-to-bottom order to obtain a single-row header.

[0061] Specifically, identifying the expansion type of merged cells refers to determining their semantic structure by analyzing the row and column span characteristics of merged cells. This can be achieved by detecting the ratio between row and column spans or by matching based on a preset merging pattern library to identify the merging logic of merged cells. Expanding merged cells by row means splitting cells that are merged across rows into independent rows. This can be achieved by retaining the original content in only the first row and setting subsequent rows to null values, avoiding redundant information caused by repeated content filling in row merging scenarios. Expanding merged cells by column means splitting cells that are merged across columns into independent columns. This can be achieved by copying the original content to all split columns to solve the problem of missing header information in column merging scenarios. Concatenating multi-row headers means linearizing multi-level headers within a single column. This can be achieved by using string concatenation operations and adding delimiters to transform hierarchical headers into a linear structure that can be directly associated with data columns.

[0062] For example, when processing the original header containing merged cells, the system first detects that a merged cell has a row span of 2 and a column span of 1, and identifies it as a row-expanded type. Then, the merged cell is split into two independent row cells, retaining only the "Quarter" text content in the first row and setting the second row to null. For another merged cell with a row span of 1 and a column span of 3, the system identifies it as a column-expanded type, splits it into three independent column cells, and copies the "Department" text content into each of the split cells. After all merged cells are expanded, for the "Year-Quarter" double-layer header in the first column, the system concatenates them from top to bottom into a "Year-Quarter" single-row header. For the "Department-Branch" double-layer header in the second column, it concatenates them into a "Department-Branch" single-row header, finally forming a complete single-row header sequence.

[0063] In this embodiment, the expansion type of merged cells is first identified to provide a basis for subsequent differentiated processing, ensuring that appropriate expansion strategies are adopted for row merging and column merging scenarios. When the expansion type is determined to be row-by-row, the content of the first row is retained and the content of subsequent rows is cleared, avoiding semantic confusion caused by content duplication in row merging scenarios. When the expansion type is determined to be column-by-column, the content is copied to each column, ensuring that all data columns in column merging scenarios can be associated with complete header information. Finally, for the multi-row headers of each column after expansion processing, the content is concatenated in a top-to-bottom order, preserving the hierarchical logical relationship of the original headers and generating a single-row linear structure that meets the requirements of subsequent parsing. This effectively avoids the problem of content loss or duplication due to misjudgment of expansion type during the compression of merged cells, ensuring the semantic integrity of header information, thus providing a structured foundation for accurate division of data columns and reliable extraction of primary key columns, and significantly improving the accuracy of complex table parsing.

[0064] S34B. Based at least the fourth prompt template and the single-row header, construct the third sub-prompt and input it into the first large language model to obtain the primary key column of the data block.

[0065] For example, the fourth prompt template is: "Please determine the first X columns that need to be included in each sub-table in order to split a table into multiple sub-tables while ensuring that the information is not cluttered. Please only answer the first X columns, and use..." <data>X< / data> Wrap the label. The complete header is: "By embedding the single-row header into the fourth prompt template, we obtain the third sub-prompt and input it into the first major language model to get the primary key column of this data block."

[0066] S35B. Determine the data columns of the data block based on the primary key column and the table range of the data block.

[0067] For example, in a personnel arrangement table, the column that begins with "Department," "Team," and "Module" is the primary key column, and the subsequent daily personnel arrangements are the data columns. Based on the primary key column of the data block and the table range, the data columns of that data block can be determined.

[0068] In some embodiments, after determining the data columns of the data block in S35B, the method further includes: If the number of columns in a data block exceeds a preset column threshold, the data block is split into multiple sub-data column groups according to a preset specified number of columns.

[0069] Based on the principle of retaining the primary key column, each sub-data column group is combined with the primary key column to generate various sub-tables.

[0070] Iterate through each sub-table, extract the header fields of each row and the cell content of that row in the data column, and obtain the standardized text of each sub-table based on the primary key column to generate the parsing result of the data block.

[0071] Specifically, the preset column number threshold refers to the critical value used to determine whether to start the table splitting process. It can be implemented using a fixed value configured by the system or a threshold dynamically calculated based on hardware resources. The purpose is to avoid introducing unnecessary processing overhead when the number of columns is moderate. The preset specified column number refers to the maximum number of columns that each sub-data column group should contain. It can be implemented through user-defined parameters or values ​​that are automatically adapted based on the processing capabilities of the large language model. The purpose is to ensure that the size of the sub-tables is controllable to meet subsequent retrieval needs. The primary key column refers to the feature column used to uniquely identify the table rows. It can be represented as a department name column, team identifier column, or module number column, etc., which have the ability to distinguish entities. The purpose is to serve as the core anchor point for data association to maintain the contextual relationship between each sub-table after splitting and the original data.

[0072] For example, when parsing an annual duty roster containing 365 columns, the system first identifies the "department" column as the primary key column. When the total number of data columns exceeds the preset threshold of 50 columns, the system splits the data columns into 18 sub-data column groups according to the specified number of columns (20). Then, each sub-data column group is combined with the "department" primary key column to generate 18 sub-tables. For example, the first sub-table contains the "department" column and the date column from January 1st to January 20th. When traversing each sub-table, the system extracts the "department" field value and the duty personnel information of the corresponding date column row by row to generate standardized text such as "R&D Department: Zhang San on January 1st; Li Si on January 2nd". Finally, the standardized text of all sub-tables is concatenated to form a complete parsing result.

[0073] This embodiment triggers conditional branching by determining whether the number of data columns exceeds a preset threshold. When the condition is met, the data column is evenly divided into multiple sub-data column groups according to the specified number of columns. Then, using the primary key column as a fixed benchmark, each sub-data column group is horizontally concatenated with the primary key column to form an independent sub-table, thereby achieving structured splitting at the column level while preserving unique row-level identifiers. Based on this, a row-level traversal operation is performed on each sub-table, simultaneously extracting the header fields and corresponding cell content. The extracted results are then organized into a standardized text format based on the identifier information of the primary key column. Finally, the original data block is parsed and output by aggregating the standardized text of all sub-tables. The pervasive retention mechanism of the primary key column ensures the integrity of row-level semantics during the splitting process of extremely long tables, while transforming large-scale column data into multiple sub-table units adapted to the processing capabilities of retrieval systems. This effectively solves the problem of information redundancy caused by direct parsing of extremely long tables, significantly improving the retrieval efficiency of table data in the document library. Furthermore, the mandatory retention mechanism of the primary key column ensures that each sub-table can still accurately trace the original data entity after splitting, thus maintaining the integrity and searchability of the complex table parsing results.

[0074] S36B. If the number of columns in the data block does not exceed the preset column number threshold, extract the header field of each row and the cell content of the row in the data column, and obtain the parsing result of the data block based on the primary key column of the data block.

[0075] Specifically, the parsing results of the data blocks are output in the format of "header1: content1; header2: content2".

[0076] Figure 3 A flowchart of a document parsing method according to another embodiment of this application is shown. (Refer to...) Figure 3 One embodiment of this application provides a document parsing method, including: S100: Read the raw OpenXML data of the Excel document to be parsed.

[0077] Specifically, for the Excel document to be parsed, the original OpenXML data of the Excel document is first read using existing software libraries in Python, such as openpyxl. The original OpenXML data of the Excel document to be parsed is read in the form of an array.

[0078] S200. For each sheet in the Excel document to be parsed, construct a third prompt based on the original OpenXML data and the preset fifth prompt template, and input it into the first major language model to obtain the table area range and cell content of each independent table in the sheet.

[0079] Specifically, in some scenarios, tabulators might place multiple independent tables in the same sheet instead of creating separate sheets. Therefore, it's first necessary to identify if this situation exists, and if so, what the respective ranges are. Since the large language model has good support for Markdown format, the first N rows (e.g., N=10) of the Excel document to be parsed are converted to Markdown format, and a preset fifth prompt template is used: "Please identify all the independent tables from the following table content. For each table, output A=top-left row number; B=top-left column number; C=total row number; D=total column number. Note that only these contents should be answered, and for each independent table..." <data> A, B, C, D< / data> Wrap the tags. The table's Markdown format content is: ".". Embed the Markdown format data of each sheet at the end of the preset fifth prompt template to obtain the table area range and cell content of each independent table.

[0080] S300. For each independent table, construct the fourth prompt based on the preset sixth prompt template and input it into the first language model to obtain the original header content of each independent table.

[0081] Specifically, in complex tables, the header often exceeds one row and contains various cell merging patterns. Therefore, it's necessary to follow the preset sixth prompt template: "Please determine the first X rows as headers and the data starting from row X+1 in the following table content. Note that only the first X rows should be answered, and use..." <data>X< / data> Wrap the text in a tag. The table's Markdown format is: ".". The original header content of each individual table is obtained by embedding the cell content of each individual table into the end of the preset sixth prompt template.

[0082] S400. According to the preset header compression rules, compress the original header content of each independent table into a single-line header.

[0083] In some embodiments, when merged cells exist in the original header content, the expansion type of the merged cells is identified.

[0084] S410. When the merged cell expansion type is set to expand by row, expand the merged cells by row and retain the content of the first row of merged cells.

[0085] S420. When the merged cell expansion type is set to expand by column, expand the merged cell by column and copy the contents of the merged cell to each column.

[0086] S430. For each column of multi-row headers, concatenate the contents of each expanded row of cells in a top-to-bottom order to obtain a single-row header.

[0087] Specifically, identifying the expansion type of merged cells refers to determining their semantic structure by analyzing the row and column span characteristics of merged cells. This can be achieved by detecting the ratio between row and column spans or by matching based on a preset merging pattern library to identify the merging logic of merged cells. Expanding merged cells by row means splitting cells that are merged across rows into independent rows. This can be achieved by retaining the original content in only the first row and setting subsequent rows to null values, avoiding redundant information caused by repeated content filling in row merging scenarios. Expanding merged cells by column means splitting cells that are merged across columns into independent columns. This can be achieved by copying the original content to all split columns to solve the problem of missing header information in column merging scenarios. Concatenating multi-row headers means linearizing multi-level headers within a single column. This can be achieved by using string concatenation operations and adding delimiters to transform hierarchical headers into a linear structure that can be directly associated with data columns.

[0088] For example, when processing the original header containing merged cells, the system first detects that a merged cell has a row span of 2 and a column span of 1, and identifies it as a row-expanded type. Then, the merged cell is split into two independent row cells, retaining only the "Quarter" text content in the first row and setting the second row to null. For another merged cell with a row span of 1 and a column span of 3, the system identifies it as a column-expanded type, splits it into three independent column cells, and copies the "Department" text content into each of the split cells. After all merged cells are expanded, for the "Year-Quarter" double-layer header in the first column, the system concatenates them from top to bottom into a "Year-Quarter" single-row header. For the "Department-Branch" double-layer header in the second column, it concatenates them into a "Department-Branch" single-row header, finally forming a complete single-row header sequence.

[0089] In this embodiment, the expansion type of merged cells is first identified to provide a basis for subsequent differentiated processing, ensuring that appropriate expansion strategies are adopted for row merging and column merging scenarios. When the expansion type is determined to be row-by-row, the content of the first row is retained and the content of subsequent rows is cleared, avoiding semantic confusion caused by content duplication in row merging scenarios. When the expansion type is determined to be column-by-column, the content is copied to each column, ensuring that all data columns in column merging scenarios can be associated with complete header information. Finally, for the multi-row headers of each column after expansion processing, the content is concatenated in a top-to-bottom order, preserving the hierarchical logical relationship of the original headers and generating a single-row linear structure that meets the requirements of subsequent parsing. This effectively avoids the problem of content loss or duplication due to misjudgment of expansion type during the compression of merged cells, ensuring the semantic integrity of header information, thus providing a structured foundation for accurate division of data columns and reliable extraction of primary key columns, and significantly improving the accuracy of complex table parsing.

[0090] S500: Based at least the preset seventh prompt template and the single-row header of each independent table, construct the fifth prompt and input it into the first large language model to obtain the primary key column of each independent table.

[0091] For example, some Excel spreadsheets may have a large number of columns, requiring splitting for subsequent data processing. For instance, a duty roster might have each row representing a department, team, or module, with hundreds of columns in a single sheet storing a year's worth of duty personnel assignments. In this case, the columns must first be split by month or week; otherwise, subsequent content retrieval will be significantly difficult. When splitting columns, the key is to identify the columns beginning with "department," "team," or "module" as primary key columns, and the subsequent daily personnel assignments as data columns. Therefore, the preset seventh prompt template is used: "Please determine the first X columns that need to be included in each sub-table to split a table into multiple sub-tables while ensuring the information remains organized. Please only answer with the first X columns, and use..." <data>X< / data> Wrap the tags. The complete table header is: ".". By embedding the single-row header of each independent table into the preset seventh prompt template, the fifth prompt is obtained, and the primary key column of each independent table is obtained after inputting the first major language model.

[0092] S600. Determine the data columns for each independent table based on the primary key column and the table range for each independent table.

[0093] S700: If the number of columns in the data column of each independent table does not exceed the preset column number threshold, extract the header field of each row and the cell content of that row in the data column, and output each independent table according to the primary key column of each independent table to obtain the parsing result of the Excel document to be parsed.

[0094] For example, the parsing result of the Excel document to be parsed is output in the format "Header 1: Content 1; Header 2: Content 2".

[0095] In some embodiments, after determining the data columns for each individual table, the method further includes: If the number of columns in the data block exceeds a preset column threshold, the data columns of each independent table are split according to a preset specified number of columns to obtain multiple sub-data column groups.

[0096] Based on the principle of retaining the primary key column, each sub-data column group is combined with the primary key column to generate various sub-tables.

[0097] Iterate through each sub-table, extract the header fields of each row and the cell content of that row in the data column, and obtain the standardized text of each sub-table based on the primary key column of each sub-table to generate the parsing result of the data block.

[0098] Specifically, the preset column number threshold refers to the critical value used to determine whether to start the table splitting process. It can be implemented using a fixed value configured by the system or a threshold dynamically calculated based on hardware resources. The purpose is to avoid introducing unnecessary processing overhead when the number of columns is moderate. The preset specified column number refers to the maximum number of columns that each sub-data column group should contain. It can be implemented through user-defined parameters or values ​​that are automatically adapted based on the processing capabilities of the large language model. The purpose is to ensure that the size of the sub-tables is controllable to meet subsequent retrieval needs. The primary key column refers to the feature column used to uniquely identify the table rows. It can be represented as a department name column, team identifier column, or module number column, etc., which have the ability to distinguish entities. The purpose is to serve as the core anchor point for data association to maintain the contextual relationship between each sub-table after splitting and the original data.

[0099] For example, when parsing an annual duty roster containing 365 columns, the system first identifies the "department" column as the primary key column. When the total number of data columns exceeds the preset threshold of 50 columns, the system splits the data columns into 18 sub-data column groups according to the specified number of columns (20). Then, each sub-data column group is combined with the "department" primary key column to generate 18 sub-tables. For example, the first sub-table contains the "department" column and the date column from January 1st to January 20th. When traversing each sub-table, the system extracts the "department" field value and the duty personnel information of the corresponding date column row by row to generate standardized text such as "R&D Department: Zhang San on January 1st; Li Si on January 2nd". Finally, the standardized text of all sub-tables is concatenated to form a complete parsing result.

[0100] This embodiment triggers conditional branching by determining whether the number of data columns exceeds a preset threshold. When the condition is met, the data column is evenly divided into multiple sub-data column groups according to the specified number of columns. Then, using the primary key column as a fixed benchmark, each sub-data column group is horizontally concatenated with the primary key column to form an independent sub-table, thereby achieving structured splitting at the column level while preserving unique row-level identifiers. Based on this, a row-level traversal operation is performed on each sub-table, simultaneously extracting the header fields and corresponding cell content. The extracted results are then organized into a standardized text format based on the identifier information of the primary key column. Finally, the original data block is parsed and output by aggregating the standardized text of all sub-tables. The pervasive retention mechanism of the primary key column ensures the integrity of row-level semantics during the splitting process of extremely long tables, while transforming large-scale column data into multiple sub-table units adapted to the processing capabilities of retrieval systems. This effectively solves the problem of information redundancy caused by direct parsing of extremely long tables, significantly improving the retrieval efficiency of table data in the document library. Furthermore, the mandatory retention mechanism of the primary key column ensures that each sub-table can still accurately trace the original data entity after splitting, thus maintaining the integrity and searchability of the complex table parsing results.

[0101] For example, Figure 4 An exemplary flowchart of a document parsing method provided in an embodiment of this application is shown. (Refer to...) Figure 4 For the Excel document to be parsed, the original data of the Excel document (i.e., the original OpenXML data of the Excel document to be parsed) is first read. The large model (i.e., the first large language model) is used to identify the table structure in the Excel document to be parsed, including the identification of independent tables, table headers, and primary key columns. Sub-table processing (i.e., independent table processing), table header processing, and splitting of excessively long tables are then performed respectively. The table data is then output row by row to obtain the parsing result of the Excel document to be parsed.

[0102] This application combines the intelligent recognition capabilities of a large language model with pre-defined rule processing. It utilizes the large model to accurately identify the regional range of multiple independent tables within a single page, parse complex header structures, and determine primary key columns. This effectively solves the problem of traditional parsing methods failing to accurately extract structured data when dealing with complex headers, irregularly merged cells, single-page segmentation, and extremely long tables, thus improving the accuracy and completeness of document parsing. Specifically, the underlying structure parsing based on raw OpenXML data avoids rendering layer distortion. Pre-defined prompt templates ensure standardized input formats to guide the large model in accurately identifying table regions and header features. Simultaneously, header compression rules achieve multi-layer header flattening through rule-based processing of merged cell logic, and the primary key column recognition mechanism ensures the integrity of core information when splitting extremely long tables. This systematically compensates for the shortcomings of traditional rule-based methods in structure recognition, significantly improving the reliability and applicability of document parsing in RAG knowledge base applications.

[0103] Figure 5 A schematic diagram of the structure of a document parsing system provided in an embodiment of this application is shown. (Refer to...) Figure 5 One embodiment of this application provides a document parsing system, including: Word document parsing system 100 is used to parse data blocks whose data type is not the target data type, so as to obtain the parsing results of data blocks whose data type is not the target data type; The first major language model 200 is used to implement any of the above document parsing methods; Document library 300 is used to store the parsing results of the Word document to be parsed obtained according to the above document parsing method, and / or the parsing results of the Excel document to be parsed obtained according to the above document parsing method; The second major language model 400 is used to search for M document slices in the document library based on the search terms input by the user, and select the first m document slices based on semantic similarity for information extraction and integration, and reply to the user with the results matching the search terms, where 1≤M, 1≤m≤M.

[0104] This application combines traditional document parsing capabilities with intelligent processing by a large language model, using a data type differentiation approach. This allows for efficient rule-based parsing of non-target data types while precise conversion of target data types (such as mathematical formulas and complex tables) using the large language model. This achieves a significant improvement in document parsing completeness and accuracy while minimizing computational resource consumption. Specifically, the Word document parsing system focuses on processing non-target data types such as text paragraphs, avoiding the use of large models for simple content and ensuring basic parsing efficiency. The first large language model constructs specific prompt inputs for target data types such as mathematical formulas, converting raw OpenXML data into standard format expressions, or identifying table ranges and header structures in Excel documents, effectively solving the problem of parsing complex structures that traditional rules cannot cover. Based on this, a document library stores structured parsing results to form a high-quality knowledge source. The second large language model dynamically filters document slices based on semantic similarity, further improving retrieval matching accuracy and response accuracy, thereby optimizing the overall performance bottleneck of the RAG knowledge base application. The first and second large language models can be the same large language model or different large language models; this application does not impose specific limitations.

[0105] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. (Refer to...) Figure 6 This application also provides an electronic device, including: processor.

[0106] Memory is used to store processor-executable instructions.

[0107] The processor is configured to execute instructions to implement any document parsing method.

[0108] In this embodiment, the computer device includes a processor, memory, and network interface connected via a system bus. The computer device's processor provides computational and control capabilities. Its memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The computer device's database stores data samples. Its network interface allows for communication with external terminals via a network connection. When executed by the processor, the computer program implements any document parsing method.

[0109] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] This application also provides a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by the processor of a terminal, enables the terminal to execute any document parsing method.

[0111] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0112] Optionally, a readable storage medium can be coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Alternatively, the readable storage medium can be an integral part of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0113] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements any document parsing method.

[0114] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0119] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0122] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.< / m:math> < / m:math> < / m:math> < / m:math> < / m:math> < / m:math> < / m:math> < / w:tbl> < / m:math> < / w:tbl> < / m:math> < / w:p> < / m:math> < / w:tbl> < / w:p> < / m:math> < / w:tbl> < / w:p> < / m:math> < / w:tbl> < / w:p>

Claims

1. A document parsing method characterized by, The method comprises: reading original OpenXML data of a Word document to be parsed; identifying each top-level tag in the original OpenXML data, taking each identified top-level tag as a data block, and determining the data type of the data block; in a case where the data type of a data block is a target data type that cannot be parsed by a Word document parsing system, constructing a first prompt according to the content of the data block and inputting the first prompt into a first large language model to obtain a parsing result of the data block; splicing the parsing result of the data block with the data type being the target data type by the first large language model and the parsing result of the data block with the data type not being the target data type by the Word document parsing system to obtain a parsing result of the Word document to be parsed.

2. The method of claim 1, wherein, The target data type at least includes mathematical formulas; The method of constructing a first prompt according to the content of the data block and inputting the first prompt into a first large language model to obtain a parsing result of the data block comprises: constructing the first prompt according to a preset first prompt template and the content of the data block, and converting the content of the data block into a standard format mathematical expression by the first large language model.

3. The method of claim 1, wherein, The target data type at least includes tables; The method of constructing a first prompt according to the content of the data block and inputting the first prompt into a first large language model to obtain a parsing result of the data block comprises: constructing a first sub-prompt according to a preset second prompt template and the content of the data block and inputting the first sub-prompt into the first large language model to obtain a table area range and cell content of the data block; constructing a second sub-prompt according to a preset third prompt template and the cell content of the data block and inputting the second sub-prompt into the first large language model to obtain original header content of the data block; compressing the original header content of the data block into a single-row header according to a preset header compression rule; constructing a third sub-prompt according to a fourth prompt template and the single-row header and inputting the third sub-prompt into the first large language model to obtain a primary key column of the data block; determining a data column of the data block according to the primary key column and the table area range of the data block; in a case where the number of columns of the data column does not exceed a preset column threshold, extracting a header field of each row and cell content of the row in the data column by row, and obtaining a parsing result of the data block according to the primary key column of the data block.

4. The method of claim 3, wherein, The method of compressing the original header content of the data block into a single-row header according to a preset header compression rule comprises: in a case where the original header content includes a merged cell, identifying an expansion type of the merged cell; in a case where the expansion type of the merged cell is row expansion, expanding the merged cell by row and retaining the content of the first row merged cell; In a case where the expansion type of the merged cell is column expansion, the merged cell is expanded by column, and the content of the merged cell is copied to each column; For each column of the multi-row table header, the content of each row of the expanded cell is spliced in a top-to-bottom order to obtain the single-row table header.

5. The method of claim 3, wherein, After determining the data column of the data block, the method further includes: In a case where the number of columns of the data column of the data block exceeds the preset column threshold, the data column of the data block is divided into a plurality of sub-data column groups according to a preset specified number of columns; Each sub-data column group is combined with the primary key column to generate a respective sub-table, with the principle of retaining the primary key column; Each sub-table is traversed, and the table header field of each row and the cell content of the row in the data column are extracted by row, and the standardized text of each sub-table is obtained according to the primary key column to generate the analysis result of the data block.

6. A document analysis method characterized by comprising: The method includes: reading original OpenXML data of an Excel document to be parsed; For each sheet of the Excel document to be parsed, a third prompt is constructed according to the original OpenXML data and a preset fifth prompt template, and a first large language model is inputted to obtain the table region range and the cell content of each independent table in the sheet; For each independent table, a fourth prompt is constructed according to a preset sixth prompt template, and the first large language model is inputted to obtain the original table header content of the each independent table; The original table header content of the each independent table is compressed into a single-row table header according to a preset table header compression rule; A fifth prompt is constructed according to at least a preset seventh prompt template and the single-row table header of the each independent table, and the first large language model is inputted to obtain the primary key column of the each independent table; The data column of the each independent table is determined according to the primary key column and the table region range of the each independent table; In a case where the number of columns of the data column of the each independent table does not exceed a preset column threshold, the table header field of each row and the cell content of the row in the data column are extracted by row, and the each independent table is outputted according to the primary key column of the each independent table to obtain the analysis result of the Excel document to be parsed.

7. A document analysis system characterized by comprising: The method includes: a Word document parsing system for parsing a data block of a data type that is not a target data type to obtain an analysis result of the data block of the data type that is not the target data type; a first large language model for implementing the document parsing method of any one of claims 1-5 or the document parsing method of claim 6; a document library for storing the analysis result of the Word document to be parsed obtained according to the document parsing method of any one of claims 1-5 and / or the analysis result of the Excel document to be parsed obtained according to the document parsing method of claim 6; A second large language model is used to find M document slices in the document library according to the search term input by the user, and the first m document slices are selected based on semantic similarity to perform information extraction and integration, and a result matching the search term is obtained to reply to the user, wherein 1≤M, 1≤m≤M.

8. An electronic device, comprising: Comprise: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the document parsing method of any one of claims 1-5, or implement the document parsing method of claim 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer readable storage medium are executed by the processor of the terminal, the terminal can execute the document parsing method of any one of claims 1-5, or implement the document parsing method of claim 6.

10. A computer program product, characterised in that, The computer program product comprises a computer program, which, when executed by a processor, implements the document parsing method of any one of claims 1-5, or implements the document parsing method of claim 6.

Images