A document parsing and exporting method and device based on a multi-modal large model, equipment and a storage medium
By generating task objects, rendering documents as rasterized images, and using a multimodal large model to determine the parsed text, the inefficiency in document parsing and exporting is solved, achieving stable document parsing and exporting, and improving user experience and traceability.
Patent Information
- Application Number
- CN202610903096.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-23
- Publication Date
- 2026-07-28
AI Technical Summary
In the process of document parsing and exporting based on multimodal large models, traditional methods are prone to omissions and illusions. The parsing results lack stable binding with PDF page numbers and original page images, making it difficult to explain which page the fragment comes from, resulting in low parsing and export efficiency.
By generating task objects, rendering documents as rasterized images, setting page text clues, using multimodal large models to determine the parsed text, and storing and merging document files based on confirmation instructions, including filenames, page numbers, rasterized images, and parsed text.
It improves the efficiency of document parsing and exporting, enhances the user experience, supports interactive optimization and traceability, and reduces the parsing and tracing costs on the knowledge base side.
Smart Images
Figure CN122472022A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for document parsing and exporting based on a multimodal large model. Background Technology
[0002] Currently, knowledge base and retrieval enhancement generation require segmenting long documents such as patents, market reports, and white papers into semantically complete and clearly defined fragments; these fragments need to have traceable evidence for review and error correction. However, traditional one-time full-text feeding to large models is prone to omissions and illusions; pure OCR or layout restoration is unstable for complex layouts; and the parsing results lack stable binding with PDF page numbers and original page images, making it difficult to explain which page the fragment comes from after being entered into the database.
[0003] As can be seen from the above, improving the efficiency of document parsing and exporting based on multimodal large models is an urgent problem to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for document parsing and exporting based on a multimodal large model, which can improve the efficiency of document parsing and exporting in the process of document parsing and exporting based on a multimodal large model. The specific solution is as follows: Firstly, this application provides a document parsing and export method based on a multimodal large model, including: A task object is generated based on the original filename of the document to be processed, and a task to be processed is determined based on the task object. Based on the task to be processed, the current page of the document to be processed is rendered as a rasterized image, and the text content of the current page is set as the page text cues. The text to be parsed on the current page is determined based on the first preset prompt word template, the rasterized image, and the page text clues; Get an update request to edit, modify and supplement the parsed text to be processed, and call the multimodal big model based on the update request, the parsed text to be processed and the second preset prompt word template. Then, use the multimodal big model and the parsed text to be processed to determine the target parsed text. Based on the confirmation instruction for the target parsed text, the current page is marked as confirmed, and the corresponding target parsed text and the rasterized image are stored; A document file is generated based on the merge export command, and the file name line, page number line, rasterized image, and target parsed text are sequentially written into each page marked as confirmed in the document file.
[0005] Optionally, generating a task object based on the original filename of the document to be processed, and determining the task to be processed based on the task object, includes: Determine the original filename corresponding to the document to be processed, and determine each task object corresponding to the document to be processed based on the original filename and the target requirements, and then add each task object to a preset task queue; A single executor retrieves the task objects from the preset task queue in a forward-to-back order, constructs a task to be processed based on each retrieved task object, and then returns progress information to the client based on the total number of pages in the document to be processed and the number of pages processed.
[0006] Optionally, determining the text to be parsed on the current page based on the first preset prompt word template and the rasterized image and the page text clues includes: The preset contextual prompts are set as system messages, and the output language description, the original file name, page number, the page text clues, and the encoded data of the rasterized image are combined into user messages; The system message and the user message are sent to the multimodal big model, so that the multimodal big model and the text to be parsed corresponding to the current page can be determined by the multimodal big model and the first preset prompt word template.
[0007] Optionally, the step of obtaining an update request to edit, modify, and supplement the parsed text to be processed, and then calling the multimodal large model based on the update request, the parsed text to be processed, and the second preset prompt word template, and then using the multimodal large model and the parsed text to be processed to determine the target parsed text, includes: Obtain an update request for editing, modifying, and supplementing the parsed text to be processed, determine the modified parsed text after editing and modification based on the update request, and set the obtained supplementary explanation as feedback content; An update message is constructed based on the second preset prompt word template, the modified parsed text, and the feedback content. Then, the target parsed text is generated using the multimodal large model and based on the update message.
[0008] Optionally, marking the current page as confirmed based on the confirmation instruction for the target parsed text includes: Receive a single-page confirmation instruction from the user for the current page, and mark the current page as confirmed based on the single-page confirmation instruction; Alternatively, it can receive a one-click confirmation command from the user for the current page and mark all pages in the document to be processed that were not marked as confirmed as confirmed.
[0009] Optionally, the step of generating a document file based on the merge export command, and sequentially writing a filename line including the original filename, a page number line, the rasterized image, and the target parsed text into each page marked as confirmed in the document file, includes: Based on the merge and export command, merge and export each page marked as confirmed corresponding to the original file name, and then write the preset delimiter string to each page marked as confirmed to obtain the corresponding pages to be processed. After the preset delimiter string in the page to be processed, write the file name line and page number line in sequence, and embed the rasterized image in the page to be processed to obtain the embedded page. Then, write the target parsed text in the embedded page.
[0010] Optionally, after sequentially writing the filename line including the original filename, the page number line, the rasterized image, and the target parsed text to each page marked as confirmed in the document file, the method further includes: When the target parsed text represents a foreign language as the primary language, the multimodal large model is invoked to translate the target parsed text into Chinese, resulting in a Chinese translation text. After the target parsed text is written into each page marked as confirmed, the Chinese translation text is written into each page marked as confirmed.
[0011] Secondly, this application provides a document parsing and exporting apparatus based on a multimodal large model, comprising: The pending task determination module is used to generate a task object based on the original file name of the document to be processed, determine the pending task based on the task object, render the current page of the document to be processed as a raster image based on the pending task, and set the text content of the current page as the page text cues. The unprocessed parsed text generation module is used to determine the unprocessed parsed text of the current page based on the first preset prompt word template and based on the rasterized image and the page text clues; The target parsing text determination module is used to obtain update requests for editing, modifying, and supplementing the parsing text to be processed, and to call the multimodal large model based on the update request, the parsing text to be processed, and the second preset prompt word template. Then, the target parsing text is determined using the multimodal large model and based on the parsing text to be processed. The target parsing text storage module is used to mark the current page as confirmed based on the confirmation instruction for the target parsing text, and store the corresponding target parsing text and the rasterized image; The document file generation module is used to generate a document file based on the merge export command, and to sequentially write a filename line including the original filename, a page number line, the rasterized image, and the target parsed text into each page marked as confirmed in the document file.
[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned document parsing and export method based on a multimodal large model.
[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned document parsing and exporting method based on a multimodal large model.
[0014] As can be seen from the above, before performing document parsing and export based on the multimodal large model, this application needs to generate a task object based on the original filename of the document to be processed, determine the task to be processed based on the task object, render the current page of the document to be processed as a raster image based on the task to be processed, and set the text content of the current page as the page text clue; determine the parsing text of the current page based on the first preset prompt word template, the raster image, and the page text clue; obtain update requests for editing, modifying, and supplementing the parsing text, call the multimodal large model based on the update request, the parsing text, and the second preset prompt word template, and then use the multimodal large model and the parsing text to be processed to determine the target parsing text; mark the current page as confirmed based on the confirmation instruction for the target parsing text, and store the corresponding target parsing text and raster image; generate a document file based on the merge export instruction, and write the filename line, page number line, raster image, and target parsing text sequentially to each page marked as confirmed in the document file.
[0015] Therefore, this application first needs to generate a task object based on the original filename of the document to be processed, and determine the task to be processed based on the task object. Based on the task to be processed, the current page of the document to be processed is rendered as a raster image, and the text content of the current page is set as the page text clue. Second, the parsing text to be processed on the current page is determined based on the first preset prompt word template, the raster image, and the page text clue. Then, an update request for editing, modifying, and supplementing the parsing text is obtained. Based on the update request, the parsing text, and the second preset prompt word template, the multimodal large model is called. Then, the target parsing text is determined using the multimodal large model and based on the parsing text to be processed. Furthermore, based on the confirmation instruction for the target parsing text, the current page is marked as confirmed, and the corresponding target parsing text and raster image are stored. Finally, a document file is generated based on the merge export instruction, and the filename line, page number line, raster image, and target parsing text are sequentially written to each page marked as confirmed in the document file. This improves the efficiency of document parsing and exporting based on multimodal large models, thereby enhancing the user experience. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 This is a flowchart of a document parsing and export method based on a multimodal large model disclosed in this application; Figure 2 This is a flowchart of a specific document parsing and export method based on a multimodal large model disclosed in this application; Figure 3 This is a schematic diagram of a specific product prototype disclosed in this application; Figure 4 This is a schematic diagram of a document parsing and exporting device based on a multimodal large model disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Knowledge base and retrieval enhancement generation require segmenting long documents such as patents, market reports, and white papers into semantically complete and clearly defined fragments; these fragments must include traceable evidence for review and error correction. However, traditional one-time full-text feeding to a large model is prone to omissions and illusions; pure OCR or layout restoration is unstable for complex layouts; and the parsing results lack stable binding with PDF page numbers and original page images, making it difficult to explain which page a fragment comes from after being entered into the database. Therefore, this application provides a document parsing and export method based on a multimodal large model, which can improve the efficiency of document parsing and export during the process of document parsing and export based on a multimodal large model.
[0020] See Figure 1 As shown in the figure, this invention discloses a document parsing and export method based on a multimodal large model, including: Step S11: Generate a task object based on the original filename of the document to be processed, determine the task to be processed based on the task object, render the current page of the document to be processed as a rasterized image based on the task to be processed, and set the text content of the current page as the page text cues.
[0021] In this embodiment, the overall architecture of this application embodiment is as follows: provided in the form of a locally or privately deployed Web service, including a document upload interface, an asynchronous task queue, document state persistence, a page-by-page parsing pipeline, a large model calling client, a translation service, a single-page and merged DOCX generation module, and a static resource access interface. A flowchart illustrating the document parsing and export method based on a multimodal large model is shown below. Figure 2 As shown, the product prototype diagram corresponding to the embodiment of this application is as follows: Figure 3 As shown.
[0022] Subsequently, this embodiment requires task and incremental uploads: each user upload generates an independent document identifier and task object; tasks enter a first-in-first-out queue, and processing functions are sequentially launched by a single executor (in this embodiment, one document task is processed at a time, while multiple documents are queued and waiting). The task status includes queue position, processing, completed, or failed, and returns the number of processed pages and the total number of pages to the client, used to display progress such as "page k / n".
[0023] Specifically, generating task objects based on the original filename of the document to be processed, and determining the tasks to be processed based on the task objects, may include: determining the original filename corresponding to the document to be processed, determining each task object corresponding to the document to be processed based on the original filename and the target requirements, and then adding each task object to a preset task queue; using a single executor to retrieve task objects from the preset task queue in a forward-to-back order, constructing the tasks to be processed based on each retrieved task object, and then returning progress information to the client based on the total number of pages in the document to be processed and the number of pages processed.
[0024] Step S12: Determine the text to be parsed for the current page based on the first preset prompt word template and the rasterized image and the page text clues.
[0025] In this embodiment, the application requires obtaining images and text page by page: the PDF is rendered using a PDF rendering library in the Node environment, each page is drawn onto the canvas and output as a bitmap file. At the same time, the application requires reading the text content object of the page and concatenating it into page text clues item by item, and writing the page number, image path, and text clues into the page state.
[0026] Subsequently, large-scale model parsing is performed: a multimodal large-scale model is invoked through a chat completion protocol compatible with open application interfaces. System-side messages are contextualized prompts; user-side messages include encoded embeddings of output language descriptions, original filenames, page numbers, page text, and page images. Then, the model returns parsed text, split into the main body and page summary, and then formatted into block text persistently containing fields for "filename, page number, main body, and summary".
[0027] Specifically, determining the parsed text of the current page based on the first preset prompt word template and the rasterized image and page text clues can include: setting the preset contextual prompt word as a system message, and combining the output language description, original file name, page number, page text clues, and the encoded data of the rasterized image into a user message; sending the system message and user message to the multimodal large model, so as to utilize the multimodal large model and the parsed text corresponding to the current page determined based on the first preset prompt word template.
[0028] Step S13: Obtain an update request for editing, modifying, and supplementing the parsed text to be processed, and call the multimodal large model based on the update request, the parsed text to be processed, and the second preset prompt word template. Then, use the multimodal large model and the parsed text to be processed to determine the target parsed text.
[0029] In this embodiment, interactive optimization is required: two sets of templates are provided: initial draft prompts and update prompts. Users can directly modify the parsed text on the interface, or trigger the update process, using the current editing area content as the previous version parsed and the user's supplementary explanations as feedback, requesting the model to regenerate and refresh the single-page Word document under the update prompt.
[0030] Specifically, the process involves obtaining update requests for editing, modifying, and supplementing the parsed text to be processed, invoking a multimodal large model based on the update requests, the parsed text to be processed, and a second preset prompt word template, and then using the multimodal large model to determine the target parsed text based on the parsed text to be processed. This process may include: obtaining update requests for editing, modifying, and supplementing the parsed text to be processed, determining the modified parsed text based on the update requests, and setting the obtained supplementary descriptions as feedback content; constructing an update message based on the second preset prompt word template, the modified parsed text, and the feedback content, and then using the multimodal large model to generate the target parsed text based on the update message.
[0031] Step S14: Based on the confirmation instruction for the target parsed text, mark the current page as confirmed, and store the corresponding target parsed text and the rasterized image.
[0032] In this embodiment, the present application supports single-page confirmation and one-click confirmation of all pages, and the confirmation status is involved in the logic of whether to allow merging the entire book.
[0033] Specifically, marking the current page as confirmed based on the confirmation instruction for the target parsed text can include: receiving a single-page confirmation instruction from the user for the current page, and marking the current page as confirmed based on the single-page confirmation instruction; or receiving a one-click confirmation instruction from the user for the current page, and marking all pages in the document to be processed that have not been marked as confirmed as confirmed.
[0034] Step S15: Generate a document file based on the merge export instruction, and sequentially write the file name line, page number line, rasterized image, and target parsed text into each page marked as confirmed in the document file.
[0035] In this embodiment, the structured Word document needs to be integrated with the knowledge base: when exporting single pages or merged documents, a predetermined separator string is written at the beginning of each logical block, followed by the "filename line," "page number line," a full-page screenshot, and the parsed text. It is worth noting that this format facilitates the database import program in cutting the document into blocks by separators, tracing back to the original PDF by page number, and performing sampling inspection using both image and text sources.
[0036] It is worth mentioning that the embodiments of this application are scalable. Furthermore, when performing page-level parsing, the embodiments of this application can use parallel scheduling to shorten the wall clock time. Then, for Word-like streaming documents, the embodiments of this application can replace "page" with pagination rendering diagrams or chapter units, while the remaining prompts, review and export modules remain unchanged.
[0037] Specifically, a document file is generated based on the merge export command, and filename lines including the original filename, page number lines, rasterized images, and target parsing text are sequentially written to each page marked as confirmed in the document file. This can include: merging and exporting each page marked as confirmed based on the merge export command, then writing a preset delimiter string to each page marked as confirmed to obtain the corresponding pages to be processed; writing filename lines and page number lines sequentially after the preset delimiter string in the pages to be processed, embedding rasterized images in the pages to be processed to obtain embedded pages, and then writing the target parsing text in the embedded pages.
[0038] In this embodiment, the user can select Chinese or English as the primary language for the parsed output. When the primary language is English, the embodiment calls the large model to translate the text into Chinese after parsing, and adds a Chinese comparison block to the single-page approval Word document for easy bilingual verification.
[0039] Specifically, after sequentially writing the filename line (including the original filename), page number line, rasterized image, and target parsing text to each page marked as confirmed in the document file, the process may further include: when the main language corresponding to the target parsing text representation is a foreign language, calling a multimodal large model to translate the target parsing text into Chinese to obtain Chinese translation text, and after writing the target parsing text to each page marked as confirmed, writing the Chinese translation text to each page marked as confirmed.
[0040] It is worth mentioning that the method corresponding to the embodiments of this application completes machine parsing and manual iteration within the same product closed loop, supporting manual modification, further optimization of prompt words, and batch confirmation, alleviating the problem of uncontrollable generation of professional long documents in one go; the export format comes with segment separators, file names, page numbers, and full-page screenshots, reducing the cost of re-parsing and tracing on the knowledge base side, which is conducive to improving the interpretability of vector retrieval. Furthermore, the scenario-based multi-prompt word, previous version parsing, and user feedback regeneration mechanism enable the same page to be quickly re-parsed under different business perspectives; asynchronous queues and fine-grained progress down to the page balance the operability in batch construction scenarios and the user's expected management of time consumption.
[0041] As can be seen from the above, the embodiments of this application first need to generate a task object based on the original file name of the document to be processed, and determine the task to be processed based on the task object. Based on the task to be processed, the current page of the document to be processed is rendered as a raster image, and the text content of the current page is set as the page text clue. Second, the parsing text to be processed on the current page is determined based on the first preset prompt word template, the raster image, and the page text clue. Then, an update request for editing, modifying, and supplementing the parsing text is obtained. Based on the update request, the parsing text, and the second preset prompt word template, the multimodal large model is called. Then, the target parsing text is determined using the multimodal large model and based on the parsing text to be processed. Furthermore, based on the confirmation instruction for the target parsing text, the current page is marked as confirmed, and the corresponding target parsing text and raster image are stored. Finally, a document file is generated based on the merge export instruction, and the file name line, page number line, raster image, and target parsing text are sequentially written to each page marked as confirmed in the document file. This improves the efficiency of document parsing and exporting based on multimodal large models, thereby enhancing the user experience.
[0042] Accordingly, see Figure 4 As shown, this application also provides a document parsing and exporting apparatus based on a multimodal large model, including: The pending task determination module 11 is used to generate a task object based on the original file name of the document to be processed, determine the pending task based on the task object, render the current page of the document to be processed as a raster image based on the pending task, and set the text content of the current page as the page text cues. The unprocessed parsing text generation module 12 is used to determine the unprocessed parsing text of the current page based on the first preset prompt word template and based on the rasterized image and the page text clues; The target parsing text determination module 13 is used to obtain an update request for editing, modifying and supplementing the parsing text to be processed, and to call the multimodal large model based on the update request, the parsing text to be processed and the second preset prompt word template, and then use the multimodal large model and the parsing text to be processed to determine the target parsing text; The target parsed text storage module 14 is used to mark the current page as confirmed based on the confirmation instruction for the target parsed text, and store the corresponding target parsed text and the rasterized image; The document file generation module 15 is used to generate a document file based on the merge export instruction, and to sequentially write the file name line, page number line, rasterized image and target parsed text into each page marked as confirmed in the document file.
[0043] In some specific embodiments, the task determination module 11 may specifically include: The original file name determination unit is used to determine the original file name corresponding to the document to be processed, and to determine each task object corresponding to the document to be processed based on the original file name and the target requirements, and then add each task object to a preset task queue; The pending task construction unit is used to retrieve the task objects from the preset task queue in a forward-to-back order using a single executor, construct pending tasks based on each retrieved task object, and then return progress information to the client based on the total number of pages in the pending document and the number of pages processed.
[0044] In some specific embodiments, the text generation module 12 to be processed may specifically include: The user message combination unit is used to set a preset scenario-based prompt word as a system message, and combine the output language description, the original file name, the page number, the page text clue, and the encoded data of the rasterized image into a user message; The unprocessed parsing text generation subunit is used to send the system message and the user message to the multimodal large model, so as to use the multimodal large model and the first preset prompt word template to determine the unprocessed parsing text corresponding to the current page.
[0045] In some specific embodiments, the target parsing text determination module 13 may specifically include: The update request acquisition unit is used to acquire update requests for editing, modifying, and supplementing the parsed text to be processed, so as to determine the modified parsed text after editing and modification based on the update request, and set the obtained supplementary description as feedback content. The update message construction unit is used to construct an update message based on the second preset prompt word template, the modified parsed text, and the feedback content, and then use the multimodal big model to generate the target parsed text based on the update message.
[0046] In some specific embodiments, the target parsing text storage module 14 may specifically include: A single-page confirmation instruction receiving unit is used to receive a single-page confirmation instruction issued by the user for the current page, so as to mark the current page as confirmed based on the single-page confirmation instruction; The one-click confirmation instruction receiving unit is used to receive the one-click confirmation instruction issued by the user for the current page, and to mark all pages in the document to be processed that are not marked as confirmed as confirmed.
[0047] In some specific embodiments, the document file generation module 15 may specifically include: The page to be processed determination unit is used to merge and export each page marked as confirmed based on the merge and export instruction, and then write a preset delimiter string to each page marked as confirmed to obtain the corresponding page to be processed. The embedded page acquisition unit is used to sequentially write a file name line and a page number line after the preset delimiter string in the page to be processed, embed the rasterized image in the page to be processed to obtain the embedded page, and then write the target parsed text in the embedded page.
[0048] In some specific embodiments, the document parsing and exporting device based on a multimodal large model may further include: The parsing and translation unit is used to call the multimodal large model to translate the target parsing text into Chinese when the main language corresponding to the target parsing text representation is a foreign language, to obtain Chinese translation text, and after writing the target parsing text into each page marked as confirmed, write the Chinese translation text into each page marked as confirmed.
[0049] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the document parsing and export method based on a multimodal large model disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0050] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0051] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0052] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the document parsing and export method based on a multimodal large model disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.
[0053] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned document parsing and export method based on a multimodal large model. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0054] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0055] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0056] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0057] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0058] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A document parsing and export method based on a multimodal large model, characterized in that, include: A task object is generated based on the original filename of the document to be processed, and a task to be processed is determined based on the task object. Based on the task to be processed, the current page of the document to be processed is rendered as a rasterized image, and the text content of the current page is set as the page text cues. The text to be parsed on the current page is determined based on the first preset prompt word template, the rasterized image, and the page text clues; Get an update request to edit, modify and supplement the parsed text to be processed, and call the multimodal big model based on the update request, the parsed text to be processed and the second preset prompt word template. Then, use the multimodal big model and the parsed text to be processed to determine the target parsed text. Based on the confirmation instruction for the target parsed text, the current page is marked as confirmed, and the corresponding target parsed text and the rasterized image are stored; A document file is generated based on the merge export command, and the file name line, page number line, rasterized image, and target parsed text are sequentially written into each page marked as confirmed in the document file.
2. The document parsing and export method based on a multimodal large model according to claim 1, characterized in that, The process of generating a task object based on the original filename of the document to be processed, and determining the task to be processed based on the task object, includes: Determine the original filename corresponding to the document to be processed, and determine each task object corresponding to the document to be processed based on the original filename and the target requirements, and then add each task object to a preset task queue; A single executor retrieves the task objects from the preset task queue in a forward-to-back order, constructs a task to be processed based on each retrieved task object, and then returns progress information to the client based on the total number of pages in the document to be processed and the number of pages processed.
3. The document parsing and export method based on a multimodal large model according to claim 1, characterized in that, The step of determining the text to be parsed on the current page based on the first preset prompt word template and the rasterized image and the page text clues includes: The preset contextual prompts are set as system messages, and the output language description, the original file name, page number, the page text clues, and the encoded data of the rasterized image are combined into user messages; The system message and the user message are sent to the multimodal big model, so that the multimodal big model and the text to be parsed corresponding to the current page can be determined by the multimodal big model and the first preset prompt word template.
4. The document parsing and export method based on a multimodal large model according to claim 1, characterized in that, The process of obtaining an update request for editing, modifying, and supplementing the parsed text to be processed, calling the multimodal large model based on the update request, the parsed text to be processed, and the second preset prompt word template, and then using the multimodal large model and the parsed text to be processed to determine the target parsed text includes: Obtain an update request for editing, modifying, and supplementing the parsed text to be processed, determine the modified parsed text after editing and modification based on the update request, and set the obtained supplementary explanation as feedback content; An update message is constructed based on the second preset prompt word template, the modified parsed text, and the feedback content. Then, the target parsed text is generated using the multimodal large model and based on the update message.
5. The document parsing and export method based on a multimodal large model according to claim 1, characterized in that, The step of marking the current page as confirmed based on the confirmation instruction for the target parsed text includes: Receive a single-page confirmation instruction from the user for the current page, and mark the current page as confirmed based on the single-page confirmation instruction; Alternatively, it can receive a one-click confirmation command from the user for the current page and mark all pages in the document to be processed that were not marked as confirmed as confirmed.
6. The document parsing and export method based on a multimodal large model according to claim 1, characterized in that, The process of generating a document file based on the merge export command, and sequentially writing a filename line including the original filename, a page number line, the rasterized image, and the target parsed text to each page marked as confirmed in the document file, includes: Based on the merge and export command, merge and export each page marked as confirmed corresponding to the original file name, and then write the preset delimiter string to each page marked as confirmed to obtain the corresponding pages to be processed. After the preset delimiter string in the page to be processed, write the file name line and page number line in sequence, and embed the rasterized image in the page to be processed to obtain the embedded page. Then, write the target parsed text in the embedded page.
7. The document parsing and export method based on a multimodal large model according to claim 1, characterized in that, After sequentially writing the filename line, including the original filename, the page number line, the rasterized image, and the target parsed text to each page marked as confirmed in the document file, the method further includes: When the target parsed text represents a foreign language as the primary language, the multimodal large model is invoked to translate the target parsed text into Chinese, resulting in a Chinese translation text. After the target parsed text is written into each page marked as confirmed, the Chinese translation text is written into each page marked as confirmed.
8. A document parsing and exporting device based on a multimodal large model, characterized in that, include: The pending task determination module is used to generate a task object based on the original file name of the document to be processed, determine the pending task based on the task object, render the current page of the document to be processed as a raster image based on the pending task, and set the text content of the current page as the page text cues. The unprocessed parsed text generation module is used to determine the unprocessed parsed text of the current page based on the first preset prompt word template and based on the rasterized image and the page text clues; The target parsing text determination module is used to obtain update requests for editing, modifying, and supplementing the parsing text to be processed, and to call the multimodal large model based on the update request, the parsing text to be processed, and the second preset prompt word template. Then, the target parsing text is determined using the multimodal large model and based on the parsing text to be processed. The target parsing text storage module is used to mark the current page as confirmed based on the confirmation instruction for the target parsing text, and store the corresponding target parsing text and the rasterized image; The document file generation module is used to generate a document file based on the merge export command, and to sequentially write a filename line including the original filename, a page number line, the rasterized image, and the target parsed text into each page marked as confirmed in the document file.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the document parsing and export method based on a multimodal large model as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store computer programs, wherein the computer programs, when executed by a processor, implement the document parsing and exporting method based on a multimodal large model as described in any one of claims 1 to 7.