Document parsing and structuring method, apparatus, device, and readable storage medium

Abstract

A document parsing and structuring processing method, device and equipment and a computer readable storage medium. In the method, a target parsing mode is determined by obtaining a to-be-parsed document and according to a document attribute of the to-be-parsed document; the to-be-parsed document is converted into a target format text based on the target parsing mode; and structured data of the to-be-parsed document is obtained based on a multi-round reasoning model and the target format text. The parsing mode is determined by adapting the document attribute, the document is converted into a unified target format, and the structured data is obtained in combination with the multi-round reasoning model, so that the parsing efficiency and accuracy are taken into account, the document parsing adaptability is improved, and the document parsing standardization is ensured.

Description

Technical Field

[0001] This application relates to the field of document processing technology, specifically to a document parsing and structuring method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Currently, electronic documents are widely used in government affairs, office work, scientific research and other scenarios. They come in various formats and contain multimodal content. The industry has an increasingly strong demand for efficient document parsing, accurate extraction and standardized governance.

[0003] In related technologies, document parsing and model invocation are often carried out using single-tool text extraction, traditional optical character recognition (OCR) parsing, fixed process processing, or simple segmentation.

[0004] However, existing technologies have poor multimodal compatibility, poor long document segmentation performance, and rigid coupling and poor scalability, making it difficult to meet the needs of practical applications. Summary of the Invention

[0005] This application provides a document parsing and structuring method, apparatus, device, and computer-readable storage medium, which can solve the technical problem of poor document parsing effect in the prior art.

[0006] In a first aspect, embodiments of this application provide a document parsing and structuring method, the document parsing and structuring method comprising: Obtain the document to be parsed, and determine the target parsing mode based on the document attributes of the document to be parsed. The document attributes of the document to be parsed include the editability attribute of the document to be parsed. If the document attributes of the document to be parsed are editable, the target parsing mode is determined to be a high real-time mode; if the document attributes of the document to be parsed are non-editable, the target parsing mode is determined to be a high-resolution effect mode. Based on the target parsing mode, the document to be parsed is converted into target format text. If the target parsing mode is a high real-time mode, the document to be parsed is converted into target format text using a parallel thread approach. If the target parsing mode is a high-resolution mode, the document to be parsed is converted into PDF format using the parsing engine, and then the document understanding model is called to convert the PDF format into target format text. Based on the multi-round inference model and the target format text, the structured data of the document to be parsed is obtained, specifically including the following steps: The target format text is split into several text fragments based on preset splitting rules; Create a session instance corresponding to the document to be parsed, and then initialize the context state of the session instance; For each text segment, the text segment is input into the multi-turn inference model to obtain the local structured result corresponding to each text segment, and the context state of the session instance is updated. Based on the context state, the local structured results corresponding to each text fragment are summarized and verified to generate the structured data of the document to be parsed.

[0007] In conjunction with the first aspect, in one implementation, splitting the target format text into several text fragments based on preset splitting rules includes: The text length of the target format text is determined. If the text length is greater than the preset text length, the semantic integrity boundary in the target format text is determined according to the semantic analysis method. The semantic integrity boundary includes paragraph end marks, heading level marks, or sentence end marks. The target format text is segmented into several text fragments based on the semantic integrity boundary.

[0008] In conjunction with the first aspect, in one implementation, if the text length is less than or equal to a preset text length, the target format text is treated as a text segment.

[0009] In conjunction with the first aspect, in one implementation, the document parsing and structuring method further includes: Obtain the business type of the document to be parsed, and determine prompt words based on the business type, wherein the prompt words contain structured instructions for constraining the output format; The prompt words are combined with the text fragments and input into the multi-round inference model to obtain the local structured results corresponding to each text fragment.

[0010] Secondly, embodiments of this application provide a document parsing and structuring processing apparatus, the document parsing and structuring processing apparatus comprising: The determination module is used to acquire the document to be parsed and determine the target parsing mode based on the document attributes of the document to be parsed. The document attributes of the document to be parsed include the editability attribute of the document to be parsed. If the document attributes of the document to be parsed are editable, the target parsing mode is determined to be a high real-time mode; if the document attributes of the document to be parsed are non-editable, the target parsing mode is determined to be a high-resolution effect mode. The conversion module is used to convert the document to be parsed into target format text based on the target parsing mode. If the target parsing mode is a high real-time mode, the document to be parsed is converted into target format text using a thread parallel method. If the target parsing mode is a high-resolution effect mode, the document to be parsed is converted into PDF format using the parsing engine, and then the document understanding model is called to convert the PDF format into target format text. The module obtains structured data of the document to be parsed based on a multi-round inference model and the target format text, specifically including the following steps: The target format text is split into several text fragments based on preset splitting rules; Create a session instance corresponding to the document to be parsed, and then initialize the context state of the session instance; For each text segment, the text segment is input into the multi-turn inference model to obtain the local structured result corresponding to each text segment, and the context state of the session instance is updated. Based on the context state, the local structured results corresponding to each text fragment are summarized and verified to generate the structured data of the document to be parsed.

[0011] Thirdly, embodiments of this application provide a document parsing and structuring processing device, the document parsing and structuring processing device including a processor, a memory, and a document parsing and structuring processing program stored in the memory and executable by the processor, wherein when the document parsing and structuring processing program is executed by the processor, it implements the steps of the document parsing and structuring processing method as described in the first aspect.

[0012] Fourthly, embodiments of this application provide a computer-readable storage medium storing a document parsing and structuring processing program, wherein when the document parsing and structuring processing program is executed by a processor, it implements the steps of the document parsing and structuring processing method as described in the first aspect.

[0013] The beneficial effects of the technical solutions provided in this application include: By acquiring the document to be parsed and determining the target parsing mode based on its document attributes, the document to be parsed is converted into target format text based on the target parsing mode. Based on a multi-round inference model and the target format text, structured data of the document to be parsed is obtained. The parsing mode is determined by adapting the document attributes, and the document is converted into a unified target format. By combining the multi-round inference model to obtain structured data, the parsing efficiency and accuracy are balanced, improving document parsing adaptability and ensuring document parsing standardization. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating an embodiment of the document parsing and structured processing method of this application; Figure 2 For this application Figure 1 A detailed flowchart of step S20; Figure 3 For this application Figure 1 A detailed flowchart of step S30; Figure 4 This is a schematic diagram of the functional modules of an embodiment of the document parsing and structuring processing apparatus of this application; Figure 5 This is a schematic diagram of the hardware structure of the document parsing and structuring processing device involved in the embodiments of this application. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0017] In a first aspect, embodiments of this application provide a document parsing and structuring method.

[0018] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the document parsing and structuring method of this application. Figure 1 As shown, document parsing and structuring methods include: Step S10: Obtain the document to be parsed, and determine the target parsing mode based on the document attributes of the document to be parsed; In this embodiment, a differential matching and parsing strategy based on the document's own characteristics is used to reasonably select the corresponding parsing mode, ensuring the adaptability of subsequent document processing.

[0019] Further, in one embodiment, the document attributes of the document to be parsed include the editability attribute of the document to be parsed, and determining the target parsing mode based on the document attributes of the document to be parsed includes: If the document attribute of the document to be parsed is editable, then the target parsing mode is determined to be a high real-time mode; If the document attribute of the document to be parsed is non-editable, then the target parsing mode is determined to be the high-resolution effect mode.

[0020] In this embodiment, when the document to be parsed is in a directly editable format such as Word or TXT, its document attribute is determined to be an editable format, and the target parsing mode is determined to be the high real-time mode to prioritize parsing speed; when the document to be parsed is in a non-editable format such as a scanned document or image, its document attribute is determined to be a non-editable format, and the target parsing mode is determined to be the high-resolution effect mode to prioritize parsing accuracy.

[0021] In addition, the high real-time mode and high-resolution effect mode can be preset according to the user's own selection.

[0022] Step S20: Based on the target parsing mode, convert the document to be parsed into target format text; In this embodiment, the document content is uniformly and standardized according to the matched parsing pattern, and the output is a standardized text with a unified and usable format.

[0023] Furthermore, in one embodiment, converting the document to be parsed into target format text based on the target parsing mode includes: If the target parsing mode is a high real-time mode, then the document to be parsed is converted into target format text using a thread parallel approach. If the target parsing mode is a high-resolution effect mode, then the parsing engine converts the document to be parsed into PDF format, and then the document understanding model is called to convert the PDF format into the target format text.

[0024] In this embodiment, refer to Figure 2 , Figure 2 For this application Figure 1 The detailed flowchart of step S20 shows that, based on the target parsing mode determined in step S10, a document format conversion operation is performed. If the target parsing mode is a high real-time mode, a multi-threaded parallel processing method is adopted to simultaneously parse and convert different paragraphs of the editable document, quickly converting them into a unified target format text, which is rich text. If the target parsing mode is a high-resolution effect mode, the non-editable document is first converted into PDF format by the parsing engine, and then the document understanding model is called to perform high-precision parsing of the PDF document, converting it into rich text, ensuring the integrity and accuracy of the converted text.

[0025] Step S30: Based on the multi-round inference model and the target format text, obtain the structured data of the document to be parsed.

[0026] In this embodiment, multi-round reasoning capabilities are used to deeply analyze text content, sort out document logical information, and output complete, standardized, and logically coherent structured document data.

[0027] Furthermore, in one embodiment, obtaining the structured data of the document to be parsed based on the multi-turn inference model and the target format text includes: The target format text is split into several text fragments based on preset splitting rules; Create a session instance corresponding to the document to be parsed, and then initialize the context state of the session instance; For each text segment, the text segment is input into the multi-turn inference model to obtain the local structured result corresponding to each text segment, and the context state of the session instance is updated. Based on the context state, the local structured results corresponding to each text fragment are summarized and verified to generate the structured data of the document to be parsed.

[0028] In this embodiment, refer to Figure 3 , Figure 3 For this application Figure 1 The detailed flowchart of step S30 shows that, firstly, the target format text is split into several text fragments that can be processed by the model according to the preset splitting rules; then, a unique corresponding session instance is created for the current document to be parsed, and the context state of the session instance is initialized to store subsequent processing information; next, each text fragment is processed one by one, and it is input into the multi-round inference model to obtain local structured results, while the context state of the session instance is updated synchronously; finally, based on the context state, all local structured results are summarized and verified, and integrated into complete and coherent structured data of the document to be parsed.

[0029] In this embodiment, the document to be parsed is obtained, and a target parsing mode is determined based on the document attributes of the document to be parsed. Based on the target parsing mode, the document to be parsed is converted into target format text. Based on the multi-round inference model and the target format text, the structured data of the document to be parsed is obtained. The parsing mode is determined by adapting the document attributes, and the document is converted into a unified target format. The structured data is obtained by combining the multi-round inference model, which balances parsing efficiency and accuracy, improves the adaptability of document parsing, and ensures the standardization of document parsing.

[0030] Furthermore, in one embodiment, the step of splitting the target format text into several text fragments based on preset splitting rules includes: The text length of the target format text is determined. If the text length is greater than the preset text length, the semantic integrity boundary in the target format text is determined according to the semantic analysis method. The semantic integrity boundary includes paragraph end marks, heading level marks, or sentence end marks. The target format text is segmented into several text fragments based on the semantic integrity boundary.

[0031] In this embodiment, the text length of the target format text is first calculated and compared with the preset text length. If the text length is greater than the preset text length, the text is semantically identified through semantic analysis to determine the semantic integrity boundary. This boundary may include paragraph end marks, heading level marks, or sentence end marks, etc., to ensure that the split does not destroy the semantics of the text. Then, based on the determined semantic integrity boundary, the target format text is divided into several semantically complete text segments.

[0032] Furthermore, in one embodiment, if the text length is less than or equal to a preset text length, the target format text is treated as a text segment.

[0033] In this embodiment, when the length of the target format text obtained by statistics is less than or equal to the preset text length, it means that the text can be directly adapted to the input requirements of the multi-round inference model. There is no need to perform splitting operations. The target format text can be directly used as a single text fragment and input into the multi-round inference model for processing, which simplifies the processing flow and improves efficiency.

[0034] Furthermore, in one embodiment, the document parsing and structuring processing method further includes: Obtain the business type of the document to be parsed, and determine prompt words based on the business type, wherein the prompt words contain structured instructions for constraining the output format; The prompt words are combined with the text fragments and input into the multi-round inference model to obtain the local structured results corresponding to each text fragment.

[0035] In this embodiment, the business type of the document to be parsed is first obtained, such as financial statements or contract documents. Based on the requirements of different business types, corresponding prompt words are determined. These prompt words contain structured instructions that constrain the output format of the model. When processing each text fragment, the prompt words are combined with the text fragment and then input into the multi-round inference model to ensure that the local structured results output by the model meet the format requirements of the corresponding business scenario, thereby improving the standardization of structured data.

[0036] Secondly, embodiments of this application also provide a document parsing and structuring processing apparatus.

[0037] In one embodiment, reference is made to Figure 4 , Figure 4 This is a schematic diagram of the functional modules of an embodiment of the document parsing and structuring processing apparatus of this application. Figure 4 As shown, the document parsing and structuring processing device includes: The determination module 10 is used to obtain the document to be parsed and determine the target parsing mode based on the document attributes of the document to be parsed; The conversion module 20 is used to convert the document to be parsed into text in the target format based on the target parsing mode; Module 30 is used to obtain structured data of the document to be parsed based on the multi-round inference model and the target format text.

[0038] Furthermore, in one embodiment, the determining module 10 is used to: If the document attribute of the document to be parsed is editable, then the target parsing mode is determined to be a high real-time mode; If the document attribute of the document to be parsed is non-editable, then the target parsing mode is determined to be the high-resolution effect mode.

[0039] Furthermore, in one embodiment, the conversion module 20 is used to: If the target parsing mode is a high real-time mode, then the document to be parsed is converted into target format text using a thread parallel approach. If the target parsing mode is a high-resolution effect mode, then the parsing engine converts the document to be parsed into PDF format, and then the document understanding model is called to convert the PDF format into the target format text.

[0040] Furthermore, in one embodiment, the obtaining module 30 is used for: The target format text is split into several text fragments based on preset splitting rules; Create a session instance corresponding to the document to be parsed, and then initialize the context state of the session instance; For each text segment, the text segment is input into the multi-turn inference model to obtain the local structured result corresponding to each text segment, and the context state of the session instance is updated. Based on the context state, the local structured results corresponding to each text fragment are summarized and verified to generate the structured data of the document to be parsed.

[0041] Furthermore, in one embodiment, the document parsing and structuring processing apparatus further includes a splitting module, used for: The text length of the target format text is determined. If the text length is greater than the preset text length, the semantic integrity boundary in the target format text is determined according to the semantic analysis method. The semantic integrity boundary includes paragraph end marks, heading level marks, or sentence end marks. The target format text is segmented into several text fragments based on the semantic integrity boundary.

[0042] Furthermore, in one embodiment, the splitting module is also used for: If the text length is less than or equal to the preset text length, then the target format text is treated as a text fragment.

[0043] Furthermore, in one embodiment, the document parsing and structuring processing apparatus further includes an input module, used for: Obtain the business type of the document to be parsed, and determine prompt words based on the business type, wherein the prompt words contain structured instructions for constraining the output format; The prompt words are combined with the text fragments and input into the multi-round inference model to obtain the local structured results corresponding to each text fragment.

[0044] The functions of each module in the above-mentioned document parsing and structuring processing device correspond to the steps in the above-mentioned document parsing and structuring processing method embodiment, and their functions and implementation processes will not be described in detail here.

[0045] Thirdly, embodiments of this application provide a document parsing and structuring processing device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0046] Reference Figure 5 , Figure 5 This is a schematic diagram of the hardware structure of the document parsing and structuring processing device involved in the embodiments of this application. In the embodiments of this application, the document parsing and structuring processing device may include a processor, a memory, a communication interface, and a communication bus.

[0047] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0048] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the document parsing and structured processing equipment, as well as interfaces used for interconnecting the document parsing and structured processing equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0049] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0050] The processor can be a general-purpose processor, which can call the document parsing and structuring processing program stored in memory and execute the document parsing and structuring processing method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the document parsing and structuring processing program is called can be referred to in the various embodiments of the document parsing and structuring processing method of this application, and will not be repeated here.

[0051] Those skilled in the art will understand that Figure 5 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0053] The present application has a document parsing and structuring processing program stored on a computer-readable storage medium, wherein when the document parsing and structuring processing program is executed by a processor, it implements the steps of the document parsing and structuring processing method described above.

[0054] The method implemented when the document parsing and structuring processing program is executed can be referred to in the various embodiments of the document parsing and structuring processing method of this application, and will not be repeated here.

[0055] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0056] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0057] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0058] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0059] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0061] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A document parsing and structuring method, characterized in that, The document parsing and structuring processing method includes: Obtain the document to be parsed, and determine the target parsing mode based on the document attributes of the document to be parsed. The document attributes of the document to be parsed include the editability attribute of the document to be parsed. If the document attributes of the document to be parsed are editable, the target parsing mode is determined to be a high real-time mode; if the document attributes of the document to be parsed are non-editable, the target parsing mode is determined to be a high-resolution effect mode. Based on the target parsing mode, the document to be parsed is converted into target format text. If the target parsing mode is a high real-time mode, the document to be parsed is converted into target format text using a parallel thread approach. If the target parsing mode is a high-resolution mode, the document to be parsed is converted into PDF format using the parsing engine, and then the document understanding model is called to convert the PDF format into target format text. Based on the multi-round inference model and the target format text, the structured data of the document to be parsed is obtained, specifically including the following steps: The target format text is split into several text fragments based on preset splitting rules; Create a session instance corresponding to the document to be parsed, and then initialize the context state of the session instance; For each text segment, the text segment is input into the multi-turn inference model to obtain the local structured result corresponding to each text segment, and the context state of the session instance is updated. Based on the context state, the local structured results corresponding to each text fragment are summarized and verified to generate the structured data of the document to be parsed.

2. The document parsing and structuring processing method as described in claim 1, characterized in that, The step of splitting the target format text into several text fragments based on preset splitting rules includes: The text length of the target format text is determined. If the text length is greater than the preset text length, the semantic integrity boundary in the target format text is determined according to the semantic analysis method. The semantic integrity boundary includes paragraph end marks, heading level marks, or sentence end marks. The target format text is segmented into several text fragments based on the semantic integrity boundary.

3. The document parsing and structuring processing method as described in claim 2, characterized in that, If the text length is less than or equal to the preset text length, then the target format text is treated as a text fragment.

4. The document parsing and structuring processing method as described in claim 1, characterized in that, The document parsing and structuring processing method also includes: Obtain the business type of the document to be parsed, and determine prompt words based on the business type, wherein the prompt words contain structured instructions for constraining the output format; The prompt words are combined with the text fragments and input into the multi-round inference model to obtain the local structured results corresponding to each text fragment.

5. A document parsing and structuring processing apparatus, characterized in that, The document parsing and structuring processing device includes: The determination module is used to acquire the document to be parsed and determine the target parsing mode based on the document attributes of the document to be parsed. The document attributes of the document to be parsed include the editability attribute of the document to be parsed. If the document attributes of the document to be parsed are editable, the target parsing mode is determined to be a high real-time mode; if the document attributes of the document to be parsed are non-editable, the target parsing mode is determined to be a high-resolution effect mode. The conversion module is used to convert the document to be parsed into target format text based on the target parsing mode. If the target parsing mode is a high real-time mode, the document to be parsed is converted into target format text using a thread parallel method. If the target parsing mode is a high-resolution effect mode, the document to be parsed is converted into PDF format using the parsing engine, and then the document understanding model is called to convert the PDF format into target format text. The module obtains structured data of the document to be parsed based on a multi-round inference model and the target format text, specifically including the following steps: The target format text is split into several text fragments based on preset splitting rules; Create a session instance corresponding to the document to be parsed, and then initialize the context state of the session instance; For each text segment, the text segment is input into the multi-turn inference model to obtain the local structured result corresponding to each text segment, and the context state of the session instance is updated. Based on the context state, the local structured results corresponding to each text fragment are summarized and verified to generate the structured data of the document to be parsed.

6. A document parsing and structuring processing device, characterized in that, The document parsing and structuring processing device includes a processor, a memory, and a document parsing and structuring processing program stored in the memory and executable by the processor, wherein when the document parsing and structuring processing program is executed by the processor, it implements the steps of the document parsing and structuring processing method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a document parsing and structuring processing program, wherein when the document parsing and structuring processing program is executed by a processor, it implements the steps of the document parsing and structuring processing method as described in any one of claims 1 to 4.

Images