Method and device for automatically generating PPT (Power Point) from existing document based on multi-modal large model

Abstract

The invention discloses a method and a device for automatically generating a PPT (Power Point) by an existing document based on a multi-modal large model, and belongs to the field of PPT automatic generation. The method comprises the following steps: S1, constructing PPT template pages, each template page comprises a plurality of shapes, and the fillable shapes are called elements; shape names corresponding to the elements are modified according to naming requirements, and PPT template construction is completed; s2, PPT text content is generated based on the document, document preprocessing is conducted firstly, and then the document is processed through a document self-adaptive processing mechanism; and S3, after the PPT template and the PPT text content are prepared, combining the PPT template and the PPT text content by utilizing an adaptive filling template design, filling the text content into a corresponding template position, and generating a key point content illustration. The method is not limited by the word number of the Word document and the number of the Word document, non-text information such as tables and pictures can be processed, manufacturing is easy, and the method is more attractive and reasonable.

Description

Technical Field

[0001] The present invention relates to the field of automatic PPT generation, and more specifically, to a method and device for automatically generating PPT from existing documents based on a multimodal large model. Background Art

[0002] In daily office work, making a PPT with both pictures and texts usually requires manual operations by users, including extracting content from documents, selecting and inserting appropriate images, adjusting the layout of text and images, selecting appropriate templates, etc. This process is time-consuming and labor-intensive, and has high requirements on the user's design ability and typesetting skills. Especially when it is necessary to extract content from long Word documents or multiple Word documents and integrate them into a PPT, users need to perform a large number of repetitive operations, which is prone to content duplication, inconsistency or omissions.

[0003] Existing document generation PPT solution:

[0004] 1. Automatically generate PPT based on large models and predefined templates

[0005] The solution first uses the big model to understand the semantics of the Word document submitted by the user and obtain the outline. Then, according to the title of each level of the outline, the big model is used to analyze and extract the corresponding content in the document, and finally the extracted content is filled into the corresponding position in the PPT template.

[0006] This solution has the following disadvantages:

[0007] 1) Since the solution simply utilizes the capabilities of the large model and the context length token limit of the large model, it can only process single documents with a limited number of words and is not capable of processing multiple documents;

[0008] 2) The ability to combine the content with the PPT template is poor, resulting in unsatisfactory PPT effects;

[0009] 3) The PPT template design is simple and cannot meet some production requirements, resulting in the final generated PPT not being beautiful enough and possibly lacking illustrations;

[0010] 4) In the process of analyzing and refining documents, due to the lack of special processing of non-text information, non-text information in the document, such as tables, pictures, etc., is ignored, resulting in the omission of important information in the generated PPT.

[0011] 2. Purely artificial

[0012] The solution is that the user reads the document content by himself, and then based on his own understanding, fills the content into the PPT template he finds, searches for corresponding pictures, and finally makes a PPT with both pictures and text.

[0013] This solution has the following disadvantages:

[0014] 1) When there is too much document content or too many documents, some information may be missed when making a PPT, and the key content cannot be fully extracted;

[0015] 2) Low efficiency, requiring a lot of time to make PPTs and find PPT illustrations;

[0016] 3) The production threshold is high and requires certain PPT production experience. Summary of the invention

[0017] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method and device for automatically generating PPT from existing documents based on a multimodal large model. The method and device are not limited by the number of words or the number of Word documents and can process non-text information such as tables and pictures. The method is simple to produce and more beautiful and reasonable.

[0018] The object of the present invention is achieved through the following solutions:

[0019] A method for automatically generating PPT from existing documents based on a multimodal large model comprises the following steps:

[0020] S1, construct a PPT template page, each template page includes a number of shapes, and the fillable shapes are called elements; modify the shape names corresponding to the elements according to the naming requirements to complete the PPT template construction;

[0021] S2, generating PPT text content based on the document, first preprocessing the document, and then processing the document using the document adaptive processing mechanism;

[0022] S3, when the PPT template and PPT text content are prepared, the two are combined using the adaptive filling template design, the text content is filled into the corresponding template position, and the key content illustrations are generated.

[0023] Furthermore, the PPT template page includes a home page, a directory page, a transition page, a comparison page, a discussion page and a thank you page, and the production of the corresponding template page is completed by filling the page attributes into the speech notes column corresponding to the template page.

[0024] Further, in step S2, the documents include a long Word document, a single Word document and multiple Word documents.

[0025] Furthermore, in step S3, the PPT text content is generated based on the document, and the document is preprocessed first, which specifically includes the following sub-steps:

[0026] S301, selecting a document from a document list;

[0027] S302, remove invalid characters;

[0028] S303, identifying the pictures and tables in the document and recording their locations, and saving the table and picture contents in a unified image format;

[0029] S304, using the multimodal large model to convert the non-text content into a text description. After the image is transmitted to the multimodal large model, the image content understanding capability of the multimodal large model is used to convert the non-text content into a text description.

[0030] S305, replacing the pictures and tables in the original document with the text description obtained in step S304;

[0031] S306, checking whether the document list processing is completed, if completed, then the process ends; otherwise, continue from step S301.

[0032] Furthermore, in step S3, the document is processed by reusing the document adaptive processing mechanism, specifically including the sub-steps:

[0033] S311, merge all preprocessed documents into one txt document, and calculate how many tokens N the full text of the txt document will occupy doc ;

[0034] S312, determine whether the number of tokens required is less than 0.9*the number of large model context tokens; if so, proceed to S313, otherwise proceed to S315;

[0035] S313, passing the full text of the merged document as a reference text to the large model, allowing it to generate a PPT outline containing secondary headings;

[0036] S314, traverse the first-level titles and second-level titles in the generated outline in turn, pass the merged document content as reference text, first-level titles, and second-level titles to the large model, and let it generate the key points under each second-level title in the PPT outline, and enter S320;

[0037] S315, when the large model context length cannot accommodate the full text content, the merged document is segmented into several text blocks;

[0038] S316, traverse each text block, let the large model generate the outline and section summary corresponding to the text block, merge the outline and section summary of each text block into one document to form a summary document; determine whether the content in the merged document has been processed, if not, go to step S315; if completed, go to S317;

[0039] S317, passing the full text of the summary document as a reference text to the large model, allowing it to generate a PPT outline of the entire document list;

[0040] S318, the files in the document list are transmitted to the retrieval enhancement generation RAG system, and the file content is vectorized by the RAG system and stored in the vector database; RAG transmits the relevant text to the big model, so that it generates corresponding content based on the relevant text;

[0041] S319, traverse the first-level titles and second-level titles in the generated outline in turn, pass the first-level titles and second-level titles to RAG, and generate key points under each second-level title in the PPT outline based on the RAG system;

[0042] S320, completing the generation of PPT text content.

[0043] Furthermore, in step S315, the merged document is divided into several text blocks, specifically including the sub-steps of: using the dividing line of the original document as the dividing point, and then using the paragraph as the dividing point, and the number of tokens occupied by each text block cannot exceed 0.9*large model context length.

[0044] Further, in step S3, when the PPT template and the PPT text content are prepared, the two are combined by using the adaptive filling template design, the text content is filled into the corresponding template position, and the key content illustration is generated, which specifically includes the following sub-steps:

[0045] S331, making a home page, selecting a template page whose template page attribute includes "Type: Home Page" from the PPT template, copying it to the target PPT, filling the reporter name, PPT presentation title, and date content set by the user into the text boxes named "Reporter", "Main Title", and "Date" in the template page, and completing the making of the home page;

[0046] S332, making a catalog page, selecting a template page whose template page attribute includes "type: catalog page" from the PPT template, and copying it to the target PPT;

[0047] S333, take out a first-level title i first ,i first The index of the first-level heading;

[0048] S334, making a transition page, selecting a template page whose template page attribute includes "Type: Transition Page" from the PPT template, and copying it to the target PPT; first The index and first-level title text contents are filled into the text boxes named "Chapter Number" and "Chapter Title" respectively;

[0049] S335, taking out a second-level title under the current first-level title and its corresponding key points;

[0050] S336, if the number of key points P>1, extract the key points and create a comparison page; if the number of key points <=1, extract the key points and create a discussion page;

[0051] S337, after the PPT page corresponding to the key points under the secondary title is created, if the secondary title list has not been traversed, the secondary index is increased by 1, and the process goes to step S335; if the traversal is completed, the process goes to step S338;

[0052] S338, if the first-level title list is not traversed, then i first =i first +1, go to step S333; if the traversal is completed, go to step S339;

[0053] S339, creating a thank you page, obtaining a template page whose template page attribute includes "type: thank you page" from the PPT template, and copying it to the target PPT;

[0054] S340, at this point, the production of a PPT is completed.

[0055] A device for automatically generating PPT from existing documents based on a multimodal large model comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, any of the methods described above is executed.

[0056] The beneficial effects of the present invention include:

[0057] The present invention proposes a PPT automatic generation method that can process non-text information such as tables and pictures without being restricted by the number of words or the number of Word documents.

[0058] In view of the problems existing in the existing methods, such as the crude PPT and the poor integration of content and PPT template, the method of the present invention designs a comprehensive and flexible PPT template construction scheme, which can quickly adapt the existing PPT template to the PPT template requirements of the present invention. Based on the PPT template construction scheme, an adaptive template filling scheme is proposed, which intelligently selects template pages and content filling according to the text content, thereby generating a beautiful and reasonable slide.

[0059] In view of the fact that the existing methods do not extract and analyze non-text information such as pictures and tables, resulting in the final generated PPT omitting important information, the method of the present invention identifies the table and picture information in the document, and uses a multimodal large model to analyze this information, convert it into a plain text description, and replace the table and pictures with the text description.

[0060] In view of the problem that existing methods cannot handle documents with too many words and multiple documents, the method of the present invention adopts a document adaptive processing mechanism. First, all documents are merged into one document, and the total number of words in the merged document is calculated to correspond to the number of tokens. If the number of tokens is less than the number of context tokens of the large model, all the contents of the document are passed to the large model as reference information, allowing it to generate the outline and key points of the PPT step by step. If it is greater than the number of tokens, the merged document is split into several text blocks, and an outline and a section summary are generated for each text block respectively. Then, the outline and section summary generated for each text block are merged into a large outline and section summary, and the reference information of the large outline and section summary is passed to the large model, allowing the large model to generate the final PPT outline. Finally, for each section of the outline, the retrieval enhancement technology (RAG) is used to obtain the key points of each section from the original document list. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 This is a schematic diagram of the PPT template.

[0063] Figure 2 Pre-processing process for documents;

[0064] Figure 3 Provides a process for document adaptive processing;

[0065] Figure 4 Automatically generate processes for PPT;

[0066] Figure 5 The generated PPT text content structure;

[0067] Figure 6 This is a schematic diagram of document segmentation. DETAILED DESCRIPTION

[0068] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0069] The specific implementation process of the present invention is as follows:

[0070] 1. PPT template construction rules

[0071] Most of the existing solutions for generating PPT based on large models are to generate text or image content based on large models, and then use predefined templates to fill them. In order to optimize the aesthetic effect of the final generated PPT, the present invention designs a comprehensive, flexible and simple PPT template construction solution. The PPT template is composed of Figure 1 shown.

[0072] The PPT template consists of a home page, a directory page, a transition page, a comparison page, a discussion page, and a thank you page. The present invention refers to the above pages as template pages. Each template page has page attributes. In order to facilitate the adaptation of existing templates, the present invention only requires the template maker to fill the page attributes into the speech notes column corresponding to the template page. Each template page will include a number of shapes. These shapes constitute a template page and also affect the final display effect of using the template page. The present invention calls fillable shapes elements. The template maker only needs to modify the shape name corresponding to the element according to the naming requirements to meet the template construction requirements. The function and usage specifications of each template page are described in detail below.

[0073] Home page: The home page is the first page in the PPT, which mainly displays the PPT speech title, date, and reporter name. The present invention requires that the home page must have three text boxes, corresponding to the speech title, date, and reporter name. The speech title text box is named "Big Title", the date text box is named "Date", and the reporter text box is named "Reporter". The page attributes of the home page must be in the following format: Type: Home page.

[0074] Table of contents page: The table of contents page is to display the chapter structure of the entire PPT. There is only one table of contents page template in the PPT template. The present invention requires that the table of contents page is composed of a chapter title text box and a serial number text box. There can be 1 to n chapter title text boxes, and the serial number text box can be 0 or consistent with the number of chapter title text boxes. The name format of the chapter title text box is "Chapter Title i", where i is the index (between 1 and n), and the name format of the serial number text box is "Serial Number i", where i is the index (between 1 and n). The page properties of the table of contents page must be in the following format, where n is the number of chapter title text boxes: Type: Table of Contents Page, Table of Contents Item: n.

[0075] Transition page: The purpose of the transition page is to display the current chapter title in the PPT, and it is usually placed at the very beginning of the chapter content. The present invention requires that the transition page consists of a chapter number text box and a chapter title text box. The number range of the chapter number text box is [0,1], and the number range of the chapter title text box is [1,1]. The name of the chapter number text box is "Chapter Number", and the name of the chapter title text box is "Chapter Text Box". The page properties of the transition page must be in the following format: Type: Transition Page.

[0076] Thank you page: The thank you page is the last page in the PPT and is used for the ending. The template maker is free to play with it without any restrictions. The page properties of the thank you page must be in the following format: Type: Thank you page.

[0077] Comparison page: The comparison page is used to display the key points in the PPT. The present invention requires that the number of key points that can be displayed in the comparison page must be greater than or equal to 2. The comparison page consists of a page title text box, a parallel item number text box, a parallel item title text box, a parallel item picture box, and parallel item content. There is only one page title text box, named "Page Title"; the number of parallel item content text boxes ranges from [2,4], and the name format is "Parallel Item Content i", where i is the index (between 1 and 4); the number of parallel item sequence number text boxes ranges from [0,0] or is the same as the number of parallel item content text boxes, and the name format is "Parallel Item Sequence Number i", where i is the index (between 1 and 4); the number of parallel item title text boxes ranges from [0,0] or is the same as the number of parallel item content text boxes, and the name format is "Parallel Item Title i", where i is the index (between 1 and 4); the number of parallel item picture boxes ranges from [0,0] or is the same as the number of parallel item content text boxes, and the name format is "Parallel Item Picture i", where i is the index (between 1 and 4). The page properties of the comparison page must be in the following format, where "serial number", "title", and "picture" in the parallel item element parameters can be filled in depending on whether there is a parallel item serial number text box, parallel item title text box, and parallel item picture box. If so, fill in: Type: Comparison page, parallel items: 4, parallel item elements: serial number, title, picture, content.

[0078] Discussion page: The discussion page is a template page designed for when there is only one key point. The discussion page consists of a page title text box, a page content text box, and a page picture box. There is only one page title text box, named "page title"; there is only one page content text box, named "content box"; there is only one page picture box, named "picture box". The page properties of the discussion page must be in the following format, where the "picture" in the page content element parameter can be filled in based on whether there is a page picture box. If there is, fill in: Type: Discussion page, page content element: content, picture.

[0079] 2. Document preprocessing

[0080] In order to make full use of the context length of the large model and to ensure that the generated PPT does not miss important information such as tables and pictures, the present invention will pre-process the document after the user submits the document. The main flow chart of document pre-processing is as follows: Figure 2 As shown, the specific steps include:

[0081] 1) Select a document from the document list. No matter it is a single document or multiple documents, each document must be preprocessed in turn.

[0082] 2) Eliminate invalid characters. When converting documents in formats such as Word into plain text, there will be a large number of invalid characters, such as continuous spaces or line breaks, which have no practical significance for understanding the semantics of the original text, but will occupy more tokens, so they need to be eliminated. The present invention directly replaces continuous spaces or line breaks with single spaces and line breaks through simple text replacement.

[0083] 3) Detect and extract non-text content such as images and table contents. Use layout analysis tools to identify images and tables in the document and record their locations, and save the tables and image contents in a unified png format.

[0084] 4) The multimodal large model converts non-text content into text descriptions. The PNG format image is passed to the multimodal large model, and the image content understanding ability of the multimodal large model is used to convert the non-text content into text descriptions.

[0085] 5) Replace the pictures and tables in the original document with the text description obtained in the previous step.

[0086] 6) Check whether the document list processing is completed, if completed, then end. Otherwise, continue from step 1.

[0087] 3. Document Adaptive Processing Mechanism

[0088] In view of the problem that existing methods cannot handle Word documents with too much text and multiple Word documents, the present invention designs a document adaptive processing mechanism. According to different situations, different mechanisms are used to understand and analyze the document content to ensure that the large model can generate reasonable PPT outline content and key points content in any situation. The quality of the outline content determines the direction of the entire PPT content, and the generation of key points depends on the titles in the outline. In other words, we use the powerful semantic understanding ability of the large model to allow it to help us automatically generate outlines and key points. The remaining work is how to fill the generated content into the PPT template. The mechanism flow is as follows: Figure 3 As shown, the specific steps include:

[0089] 1) Perform document preprocessing on all documents.

[0090] 2) Merge all the document contents into one txt document. Copy the text content of each document into the total txt document in turn.

[0091] 3) Calculate the number of tokens Ndoc required for the merged document.

[0092] 4) Determine whether Ndoc is less than (0.9*large model context token number). The remaining 10% large model context token number is used to ensure that the large model has enough free context length to generate the target content (outline, key points).

[0093] Condition 1: Ndoc is less than (0.9*number of context tokens in the large model)

[0094] If this condition is met, then we can pass the merged document content as reference text to the large model without doing any additional operations. The remaining steps are as follows:

[0095] Step 1: Pass the merged document content as reference text to the big model, and let it generate an outline containing secondary headings. The prompt words are as follows, where {ref_text} is the reference text:

[0096] #The reference content is as follows:

[0097] {ref_text}

[0098] #Task:

[0099] Please generate a PPT outline based on <reference content>.

[0100] #The output format is as follows:

[0101] ---

[0102] 1.xxx

[0103] 1.1xxx

[0104] 2.xxx

[0105] 2.1xxx

[0106] ---

[0107] Step 2: Traverse the first-level headings and second-level headings in the generated outline in turn, and pass the merged document content as reference text, first-level headings, and second-level headings to the large model to generate key content.

[0108] The prompt words are as follows, where {ref_text} is the reference text, {chapter_title} is the first-level title, and {page_title} is the second-level title:

[0109] #The reference content is as follows:

[0110] {ref_text}

[0111] #Task:

[0112] Based on the known content, generate a text content for the section, the chapter title is {chapter_title}, and the section title is {page_title}.

[0113] #The output format is as follows:

[0114] ---

[0115] Main point title:xxx

[0116] Content:xxx

[0117] xxx must be a short sentence, no more than 50 words, and cannot contain line breaks.

[0118] ---

[0119] Condition 2: Ndoc is not less than (0.9 * number of context tokens in the large model)

[0120] If this condition is met, it means that the content of the merged document exceeds the upper limit of the content length that the large model can process. This is why most existing tools do not support multi-document generation and limit the number of words. In order to solve this problem and not modify the upper limit of the content length that the large model can process, the generated content must also be reasonable and comprehensive. The present invention uses the retrieval enhanced generation (RAG) technology to help retrieve valid content from the super-long text as reference text and generate relevant content. The remaining steps are as follows:

[0121] Step 1: Take a text block from the merged document list. The length of the text block should be obtained in paragraphs as much as possible.

[0122] Step 2: Pass the text block as a reference text to the large model, and let it generate the outline + section summary. This step is to retain the summary content of the text block for subsequent generation of the entire large outline content. The prompt words are as follows:

[0123] #The reference content is as follows:

[0124] {ref_text}

[0125] #Task:

[0126] Please generate a PPT outline based on <reference content> and include the summary content of each section title.

[0127] #The output format is as follows:

[0128] ---

[0129] 1.xxx

[0130] 1.1xxx

[0131] yyy

[0132] 2.xxx

[0133] 2.1xxx

[0134] yyy

[0135] ---

[0136] Where xxx is the title and yyy is the section summary information.

[0137] Step 3: Merge the content generated in Step 2 into the outline document. This document records the outline and section summary content of all text blocks.

[0138] Step 4: Determine whether the content in the merged document has been processed. If not, proceed to Step 1. If completed, proceed to Step 5.

[0139] Step 5: Pass the full text of the summary document as reference text to the large model to generate an outline. Because we have the summary information of all documents, and the total length of the summary information generally does not exceed the context length of the large model, a reasonable outline of the entire document list can be generated.

[0140] The prompt words are as follows, where {ref_text} is the reference text:

[0141] #The reference content is as follows:

[0142] {ref_text}

[0143] #Task:

[0144] Please generate a PPT outline based on <reference content>.

[0145] #The output format is as follows:

[0146] ---

[0147] 1.xxx

[0148] 1.1xxx

[0149] 2.xxx

[0150] 2.1xxx

[0151] ---

[0152] Step 6: Pass the files in the document list to the existing Retrieval Enhanced Generation (RAG) system. RAG will vectorize the file content and store it in the vector database, so that it can be retrieved according to the prompt words. RAG will pass the relevant text to the big model to generate corresponding content based on the relevant text.

[0153] Step 7: Traverse the first-level and second-level titles in the generated outline in turn. Pass the first-level and second-level titles to RAG and let it generate key points. The prompt words are as follows, {chapter_title} is the first-level title, and {page_title} is the second-level title:

[0154] #The reference content is as follows:

[0155] {ref_text}

[0156] #Task:

[0157] Based on the known content, generate a text content for the section, the chapter title is {chapter_title}, and the section title is {page_title}.

[0158] #The output format is as follows:

[0159] ---

[0160] Main point title:xxx

[0161] Content:xxx

[0162] xxx must be a short sentence, no more than 50 words, and cannot contain line breaks.

[0163] ---

[0164] 4. Adaptive template filling solution

[0165] The text content and PPT template of the PPT have been prepared. The next step is to combine the content with the PPT template reasonably and accurately, and finally generate a complete PPT with both pictures and text. A complete process of automatically generating PPT from existing documents is as follows: Figure 4 As shown, the specific steps include:

[0166] 1) If there is no template in the PPT template library, you need to build a PPT template. The constructed PPT template should include template pages such as the home page, transition page, directory page, discussion page, comparison page, and thank you page.

[0167] 2) Adopt the adaptive document processing mechanism to generate outline + key points. The structure of the generated content is as follows: Figure 5 As shown, it includes the first-level title, second-level title and corresponding key points.

[0168] 3) Make the homepage. Select the template page whose template page attribute contains "Type: Homepage" from the PPT template, copy it to the target PPT, and fill the user-set reporter name, PPT presentation title, date and other contents into the text boxes named "Reporter", "Headline", and "Date" in the template page to complete the homepage production.

[0169] 4) Create a table of contents page. Select a template page with the page attribute containing "Type: Table of Contents Page" from the PPT template and copy it to the target PPT. The steps are as follows:

[0170] Step 1: Get the m in "Directory Item: m" and the number of first-level headings n in the page parameters.

[0171] Step 2: Copy a template page to the target PPT, take the first-level title i (i is the index of the first-level title), and the relative index ioff within the page = 1.

[0172] Step 3: Get a text box named "Serial Number ioff", if any, fill "i" as text into the text box. Get a text box named "Chapter Title ioff", fill the text of the first-level title i into the text box.

[0173] Step 4: ioff=ioff+1, i=i+1. If i<=n and ioff<=m, go to Step 3. If i<=n and ioff>m, then ioff=1, go to Step 2. If i>n, go to Step 5.

[0174] Step 5: Delete all unfilled text boxes on the current directory page.

[0175] 5) Take out a first-level title ifirst, where ifirst is the index of the first-level title.

[0176] 6) Make a transition page. Select a template page with "Type: Transition Page" in the PPT template and copy it to the target PPT. Fill the index of the first-level title ifirst and the text content of the first-level title into the text boxes named "Chapter Number" and "Chapter Title" respectively.

[0177] 7) Take out a second-level title under the current first-level title and its corresponding key points.

[0178] 8) If the number of key points P>1, take out the key points and make a comparison page. The specific steps are described as follows:

[0179] Step 1: Set the key content index i=1, the relative index within the page ioff=1, and the number of remaining unfilled key points nP=P.

[0180] Step 2: Get the template page whose template page attribute contains "Type: Comparison Page" from the PPT template, get the m in each page attribute "Parallel Item: m", select the template page with m=nP, if there is no template page that meets the condition, select the template page whose m is closest to nP. Since there may be multiple template pages that meet the condition, randomly select a template page and copy it to the target PPT. Set the secondary title content to the text box named "Page Title".

[0181] Step 3: Take the key point content i. Get a text box named "parallel item number ioff", if any, fill "i" into the text box as text. Get a text box named "parallel item title ioff", if any, fill the title in the key point content i into the text box as text. Get a text box named "parallel item content ioff", fill the content in the key point content i into the text box.

[0182] Step 4: Get a picture frame named "parallel item picture ioff". If there is one, pass the title + content in the key content i to the multimodal large model, then let it generate the corresponding picture, and finally set the generated picture to the picture frame.

[0183] Step5: ioff=ioff+1, i=i+1. If i<=P and ioff<=m, go to Step3. If i<=P and ioff>m, it means that one template page cannot meet the filling requirements, and an additional page needs to be added, nP=nP-m, ioff=1, go to Step2. If i>P, it means that the filling is completed, go to Step6.

[0184] Step 6: Delete the unfilled text boxes and picture boxes in the last page.

[0185] 9) If the number of key points is less than or equal to 1, extract the key points and create a discussion page. The specific steps are described as follows:

[0186] Step 1: Get a template page from the PPT template whose template page attribute contains "Type: Discussion Page". Since there may be multiple template pages that meet the conditions, randomly select a template page and copy it to the target PPT. Set the secondary title content to the text box named "Page Title".

[0187] Step 2: Get the first key point content. Get a text box named "Content Box", and fill the content in the key point content as text into the text box.

[0188] Step 3: Get a picture frame named "Picture Frame". If there is one, pass the title + content in the key content to the multimodal large model, and then let it generate the corresponding picture, and finally set the generated picture to the picture frame.

[0189] 10) After the PPT page corresponding to the key points under the secondary title is created, if the secondary title list has not been traversed, the secondary index +1, then go to step 7). If the traversal is completed, then go to step 11).

[0190] 11) If the first-level title list has not been traversed, then ifirst=ifirst+1, and go to step 5). If the traversal is completed, then go to step 12).

[0191] 12) Create a thank you page. Get the template page whose template page attribute contains "Type: Thank you page" from the PPT template and copy it to the target PPT.

[0192] In other embodiments, the present invention provides an implementation example of a method for automatically generating a PPT from an existing document based on a multimodal large model, which mainly includes three steps: building a PPT template, generating PPT text content based on an existing document, and adaptively filling in the PPT template.

[0193] 1. PPT template construction example

[0194] This embodiment implements a simple, comprehensive and flexible PPT template construction method by inserting marks on speech notes and text box names. The final constructed PPT template is a .pptx file. The specific production steps are described as follows:

[0195] Step 1: Create a homepage template page. In the speech notes column, set the page property to "Type: Homepage" and modify the text box name.

[0196] Step 2: Create a table of contents page template. Set the page properties in the speech notes column: "Type: Table of Contents Page", "Table of Contents Items: 4". Modify the text box name, and keep the number of the sequence number text box and chapter title text box consistent with the number of table of contents items.

[0197] Step 3: Create a transition page template. Set the page properties in the speech notes column: "Type: Transition Page". Modify the text box name as required.

[0198] Step 4: Create a comparison page template. You can create several template pages of this type according to actual needs. Set the page properties in the speech notes column to indicate that the page can be filled with up to 4 key points: "Type: Comparison Page", "Parallel Items: 4", "Parallel Item Elements: Title, Content, Picture". Modify the text box name as required.

[0199] Step 5: Create a discussion page template page. You can create multiple template pages of this type. You need to set the page properties to: "Type: Discussion Page", "Page Content Elements: Content, Picture". Modify the text box name as required.

[0200] Step 6: Create a thank you page template. You need to set the page properties to "Type: Thank you page".

[0201] 2. Generate PPT text content based on Word document

[0202] After the PPT template library is built, the next step is to generate the text content required for the PPT, that is, the outline and key points. The structure is as follows: Figure 5 As shown. The present invention processes long Word documents, multiple Word documents and other complex Word documents through a document preprocessing mechanism + a document adaptive processing mechanism, solving the problem that existing tools have word count limitations and cannot process multiple Word documents. Figure 2 and Figure 3 As shown, the specific steps are as follows:

[0203] Step 1: Preprocess each document submitted by the user in turn and remove invalid characters. For example, replace multiple spaces "yyy xxx" and multiple line breaks "xxx\n\n\n\nyyyy" with "yyy xxx" and "xxx\nyyyy" to avoid invalid characters occupying the context length of the large model.

[0204] Step 2: Preprocess the chart content of each document submitted by the user in turn, extract the chart content, save it as an image, and let the multimodal large model convert the image into a text description. For example, for Table 1, Figure 1 The content is automatically captured and saved as table1.png. Figure 1 .png, after being processed by the multimodal large model, it is converted into "Table 1 describes xxxx", " Figure 1 The description is yyyy". Finally, replace the text description in Table 1 and Figure 1 This ensures that the information contained in the tables and pictures will not be omitted in the generated PPT text content.

[0205] Step 3: Merge all preprocessed documents into a txt document and calculate how many tokens the full text of the txt document will occupy. For example, if the total number of words in the document is 10123, the number of tokens is 5630.

[0206] Step 4: Determine whether the number of tokens required is less than 0.9*the number of context tokens of the large model. If so, proceed to Step 5, otherwise proceed to Step 7.

[0207] Step 5: Pass the full text of the merged document as a reference text to the large model and let it generate a PPT outline.

[0208] Step 6: Pass the full text of the merged document as a reference text to the big model, and let it generate the key points under each secondary heading in the PPT outline. Go to Step 12.

[0209] Step 7: When the context length of the large model cannot accommodate the full text content, the merged document is divided into several text blocks, with the original document boundary as the first dividing point, and the paragraph as the second dividing point. The number of tokens occupied by each text block cannot exceed 0.9*the context length of the large model. For example, as shown, since the content of Document 1 cannot be passed to the large model as a text block, the content of Document 1 is split into two text blocks based on paragraphs. Since the content of Document 2 can be passed to the large model as a text block, it is directly used as a text block. The example of document segmentation is as follows Figure 6 shown.

[0210] Step 8: Traverse each text block and let the big model generate the outline + subsection summary corresponding to the text block. Merge the outline + subsection summary of each text block into one document to form a summary document.

[0211] Step 9: Use the full text of the summary document as a reference text and pass it to the big model to generate a PPT outline.

[0212] Step 10: Pass the merged documents to the Retrieval Enhanced Generation (RAG) system.

[0213] Step 11: Based on the RAG system, let it generate the key points under each secondary heading in the PPT outline.

[0214] Step 12: Complete the generation of PPT text content.

[0215] 3. Adaptive filling template design

[0216] When the PPT template and PPT text content are ready, the next step is to combine the two reasonably. The adaptive filling template design of the present invention can reasonably fill the text content into the appropriate template position, and automatically generate key content pictures to ensure that the final PPT is beautiful and reasonable, and the random selection method is adopted to ensure that the style of each content page is different as much as possible. Figure 4 , the specific steps are as follows:

[0217] Step 1: The user selects a PPT template and creates a blank PPT as the target PPT.

[0218] Step 2: Make a homepage. Select a homepage template page from the PPT template and copy it to the target PPT. Set the PPT report title, reporter name, and time set by the user into the corresponding text boxes. For example, the content of the "Big Title" text box is set to "Year-end Summary Report", the content of the "Reporter" text box is set to "Zhang San", and the time is set to "2024-09-06".

[0219] Step 3: Make a table of contents page. Get the table of contents page template from the PPT template and copy it to the target PPT. Fill the first-level titles of the outline into the corresponding text boxes one by one. For example, the content of the "Chapter Title 1" text box is set to the first first-level title "Performance Review", and the content of the "Serial Number 1" text box is set to "01".

[0220] Step 4: Select a first-level title and make a transition page. Get the transition page template from the PPT template and copy it to the target PPT. Fill in the first-level title and serial number into the corresponding text box. For example, set the "Chapter Number" text box to "01" and the "Chapter Title" text box to "Performance Review".

[0221] Step 5: Get the secondary title and the corresponding key points under the primary title. According to the number of key points, get the comparison page or discussion page template page from the PPT template, copy it to the target PPT, and then fill the key points and secondary title into the corresponding text boxes. For example, if the number of key points is 2, randomly select a template page from 5 (assuming there are 5) comparison page template pages, copy it to the target PPT, set the "Page Title" text box to "Main Business Goals" (secondary title), "Parallel Item Number 1" text box to "01", "Parallel Item Title 1" text box to "Customer Satisfaction" (title of key point 1), and "Parallel Item Content 1" text box to "In 2024, customer 5-star reviews are 98%..." (the content of key point 1). Similarly, the title and content of key point 2 are filled in the text boxes of "Parallel Item Title 2" and "Parallel Item Content 2".

[0222] For example, if the number of key points is 1, randomly select a template page from the three (assuming there are three) discussion page templates, copy it to the target PPT, set the "Page Title" text box to "Main Business Objectives" (secondary title), and set the "Content Box" text box to "In 2024, customer 5-star reviews are 98%..." (the content of key point 1).

[0223] Step 6: Automatically match pictures for key points. Combine the title and content of the key points and pass them to the multimodal model to generate the corresponding pictures and replace the pictures in the picture boxes. For example, in the comparison page, take out the contents of the "parallel item title 1" text box and the "parallel item content 1" text box, and pass them to the multimodal model as reference text. After the picture is generated, replace the picture in "parallel item picture 1" with the generated picture.

[0224] Step 7: Determine whether PPT pages have been generated for all second-level titles under the current first-level title. If so, go to Step 8, otherwise go to Step 5.

[0225] Step8: Determine whether all first-level titles have been processed. If so, go to Step0, otherwise go to Step4.

[0226] Step 9: Create a thank you page. Select a thank you page template and copy it to the target PPT. This completes the creation of a PPT.

[0227] The solution of the present invention has the following advantages:

[0228] Compared with existing PPT production tools: it can process long Word documents and multiple Word documents, intelligently integrate content from multiple sources, and automatically generate PPTs; the final generated PPT is rich in pictures and texts, the content is highly consistent with the design, and the effect is better; non-text content such as charts will not be omitted; template construction is simpler and more comprehensive, and it is easier to adapt to existing templates.

[0229] Compared with purely manual PPT production: automatic image and text matching avoids the trouble of users manually searching and selecting images; intelligent content extraction and typesetting, compared with pure manual, subversively improve the efficiency of content extraction and typesetting; efficient processing of single and multiple documents, avoiding the problem of purely manual omission of document information, greatly improving document processing efficiency; low production threshold, only requires users to provide documents to generate PPT with one click.

[0230] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0231] According to one aspect of an embodiment of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations.

[0232] As another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

Claims

1. A method for automatically generating PPT from existing documents based on a multimodal large model, characterized in that: The steps include: S1, construct a PPT template page, each template page includes a number of shapes, and the fillable shapes are called elements; modify the shape names corresponding to the elements according to the naming requirements to complete the PPT template construction; S2, generating PPT text content based on the document, first preprocessing the document, and then processing the document using the document adaptive processing mechanism; S3, when the PPT template and PPT text content are prepared, the two are combined using the adaptive filling template design, the text content is filled into the corresponding template position, and the key content illustrations are generated.

2. The method for automatically generating PPT from existing documents based on a multimodal large model according to claim 1 is characterized in that: The PPT template page includes a home page, a directory page, a transition page, a comparison page, a discussion page and a thank you page. The production of the corresponding template page is completed by filling the page attributes into the speech notes column corresponding to the template page.

3. The method for automatically generating PPT from existing documents based on a multimodal large model according to claim 1 is characterized in that: In step S2, the documents include a long Word document, a single Word document and multiple Word documents.

4. The method for automatically generating PPT from existing documents based on a multimodal large model according to claim 3 is characterized in that: In step S3, the PPT text content is generated based on the document, and the document is preprocessed first, which specifically includes the following sub-steps: S301, selecting a document from a document list; S302, remove invalid characters; S303, identifying the pictures and tables in the document and recording their locations, and saving the contents of the tables and pictures in a unified image format; S304, using the multimodal large model to convert the non-text content into a text description. After the image is transmitted to the multimodal large model, the image content understanding capability of the multimodal large model is used to convert the non-text content into a text description. S305, replacing the pictures and tables in the original document with the text description obtained in step S304; S306, checking whether the document list processing is completed, if completed, then the process ends; otherwise, the process continues from step S301.

5. The method for automatically generating PPT from existing documents based on a multimodal large model according to claim 4 is characterized in that: In step S3, the document is processed by the document adaptive processing mechanism, which specifically includes the following sub-steps: S311, merge all pre-processed documents into one txt document, and calculate how many tokens N the full text of the txt document will occupy doc ; S312, determine whether the number of tokens required is less than 0.9*the number of large model context tokens; if so, proceed to S313, otherwise proceed to S315; S313, passing the full text of the merged document as a reference text to the large model, allowing it to generate a PPT outline containing secondary headings; S314, traverse the first-level titles and second-level titles in the generated outline in turn, pass the merged document content as reference text, first-level titles, and second-level titles to the large model, and let it generate the key points under each second-level title in the PPT outline, and enter S320; S315, when the large model context length cannot accommodate the full text content, the merged document is segmented into several text blocks; S316, traverse each text block, let the large model generate the outline and section summary corresponding to the text block, merge the outline and section summary of each text block into one document to form a summary document; determine whether the content in the merged document has been processed, if not, go to step S315; if completed, go to S317; S317, passing the full text of the summary document as a reference text to the large model, allowing it to generate a PPT outline of the entire document list; S318, the files in the document list are transmitted to the retrieval enhancement generation RAG system, and the file content is vectorized by the RAG system and stored in the vector database; RAG transmits the relevant text to the big model, so that it generates corresponding content based on the relevant text; S319, traverse the first-level titles and second-level titles in the generated outline in turn, pass the first-level titles and second-level titles to RAG, and generate key points under each second-level title in the PPT outline based on the RAG system; S320, completing the generation of PPT text content.

6. The method for automatically generating PPT from existing documents based on a multimodal large model according to claim 5 is characterized in that: In step S315, the merged document is divided into several text blocks, which specifically includes the sub-steps of: using the dividing line of the original document as the dividing point, and then using the paragraph as the dividing point, and the number of tokens occupied by each text block cannot exceed 0.9*large model context length.

7. The method for automatically generating PPT from existing documents based on a multimodal large model according to claim 1, characterized in that: In step S3, when the PPT template and the PPT text content are prepared, the two are combined by using the adaptive filling template design, the text content is filled into the corresponding template position, and the key content illustration is generated, which specifically includes the following sub-steps: S331, making a homepage, selecting a template page whose template page attribute includes "Type: Homepage" from the PPT template, copying it to the target PPT, filling the reporter name, PPT presentation title, and date content set by the user into the text boxes named "Reporter", "Main Title", and "Date" in the template page, and completing the making of the homepage; S332, making a catalog page, selecting a template page whose template page attribute includes "Type: Catalog Page" from the PPT template, and copying it to the target PPT; S333, take out a first-level title i first ,i first The index of the first-level heading; S334, making a transition page, selecting a template page whose template page attribute includes "Type: Transition Page" from the PPT template, and copying it to the target PPT; first The index and first-level title text contents are filled into the text boxes named "Chapter Number" and "Chapter Title" respectively; S335, taking out a second-level title under the current first-level title and its corresponding key points; S336, if the number of key points P>1, extract the key points and create a comparison page; if the number of key points <=1, extract the key points and create a discussion page; S337, after the PPT page corresponding to the key points under the secondary title is created, if the secondary title list has not been traversed, the secondary index is increased by 1, and the process goes to step S335; if the traversal is completed, the process goes to step S338; S338, if the first-level title list is not traversed, then i first =i first +1, go to step S333; if the traversal is completed, go to step S339; S339, making a thank you page, obtaining a template page whose template page attribute includes "type: thank you page" from the PPT template, and copying it to the target PPT; S340, at this point, the production of a PPT is completed.

8. A device for automatically generating PPT from existing documents based on a multimodal large model, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to any one of claims 1 to 7 is executed.

Images