Powerpoint analysis method and device based on feature extraction and large language model

Abstract

The invention provides a feature extraction and large language model-based PowerPoint analysis method and device. The method comprises the following steps of: receiving a PowerPoint; for each presentation page, calling a library function in a preset presentation file open source library, extracting feature information of each visual element in the presentation page, determining layout information of each visual element based on position information in the feature information, and combining the feature information and the layout information of each visual element to form description of the presentation page; and inputting the description of each demonstration page and a preset cue word into a large language model, performing semantic analysis according to the description under the guidance of the preset cue word, and outputting an analysis result of each demonstration page. Therefore, the description including the feature information of the visual elements and the layout information is provided for the large language model to perform semantic analysis, and the analysis effect of the system on the presentation file is improved.

Description

Technical Field

[0001] The present application relates to the technical field of presentation parsing, and in particular to a presentation parsing method and device based on feature extraction and a large language model. Background Art

[0002] Presentation parsing generally refers to the process of extracting information and data from presentations (such as PowerPoint slides, or PPTs). Existing presentation parsing techniques usually only extract text and table data, which loses a lot of contextual information, such as titles, layout positions, process order, etc. This loss of information may lead to content misunderstandings, such as reversed process order or semantic errors, which is particularly evident when dealing with PPTs containing flowcharts or complex layouts.

[0003] Therefore, there is an urgent need to propose an effective presentation parsing method. Summary of the invention

[0004] In order to solve the above problems, the present application proposes a presentation parsing method, device, computer-readable storage medium and electronic device based on feature extraction and large language model, which can realize presentation parsing with high efficiency and high performance.

[0005] In a first aspect, the present application provides a presentation parsing method based on feature extraction and a large language model, the method comprising: receiving a presentation; the presentation comprising multiple presentation pages, each presentation page comprising multiple visual elements; for each presentation page, calling a library function in a preset presentation open source library to extract feature information of each visual element in the presentation page; the feature information of a single visual element at least includes, a single visual element may include text information, position information of the visual element, and an element type of the visual element; based on the position information, determining layout information of each visual element, combining the feature information and layout information of each visual element to form a description of the presentation page; inputting the description of each presentation page and preset prompt words into a large language model, so that it performs semantic parsing according to the description under the guidance of the preset prompt words, and outputting the parsing result of each presentation page; the parsing results of all presentation pages constitute the parsing result of the presentation.

[0006] Therefore, by providing a large language model with a description including feature information of visual elements and layout information for semantic parsing, the parsing effect of the system on the presentation is improved.

[0007] In a second aspect, the present application provides a presentation parsing device based on feature extraction and a large language model, the device comprising: a receiving module, for receiving a presentation; the presentation comprises multiple presentation pages, each presentation page comprises multiple visual elements; an extraction module, for calling a library function in a preset presentation open source library for each presentation page, to extract feature information of each visual element in the presentation page; the feature information of a single visual element at least includes, a single visual element may include text information, position information of the visual element, and an element type of the visual element; based on the position information, the layout information of each visual element is determined, and the feature information and layout information of each visual element are combined to form a description of the presentation page; a parsing module, for inputting the description of each presentation page and preset prompt words into a large language model, so that it performs semantic parsing according to the description under the guidance of the preset prompt words, and outputs the parsing result of each presentation page; the parsing results of all presentation pages constitute the parsing result of the presentation.

[0008] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect or any possible implementation of the first aspect.

[0009] In a fourth aspect, the present application provides an electronic device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0010] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A presentation parsing system based on feature extraction and a large language model is provided in an embodiment of the present application;

[0013] Figure 2 It is a flow chart of a presentation parsing method based on feature extraction and a large language model provided in an embodiment of the present application;

[0014] Figure 3It is a demonstration page provided by the embodiment of the present application;

[0015] Figure 4 It is a schematic diagram of a presentation parsing device based on feature extraction and a large language model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0017] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0018] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple screen terminals refers to two or more screen terminals.

[0019] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0020] Existing presentation parsing techniques usually only extract text and table data, which loses a lot of contextual information such as title, layout position, process order, etc.

[0021] Large language models (LLMs) are a hot topic in the current field of artificial intelligence. They are trained on large amounts of text data through deep learning technology to understand and generate natural language. In the field of presentation parsing technology, they can be used to parse the semantic information included in the input document.

[0022] In view of this, the embodiment of the present application extracts the feature information of each visual element in the demonstration page for each demonstration page, determines the layout information of each visual element based on the position information in the feature information, combines the feature information and layout information of each visual element, and forms a description of the demonstration page. The description of each demonstration page is then input into the large language model, so that the large language model performs semantic analysis according to the description under the guidance of the preset prompt words, obtains the analysis results of each demonstration page, and finally obtains the analysis results of the presentation. Thus, by providing the large language model with a description including the feature information and layout information of the visual elements for semantic analysis, the system's analysis effect on the presentation is improved.

[0023] For example, Figure 1 A presentation parsing system based on feature extraction and large language model provided by an embodiment of the present application is shown. The system can be deployed on any device, apparatus, platform, server, device cluster, etc. with computing and processing capabilities.

[0024] like Figure 1 As shown, the presentation parsing system 100 includes a feature extraction module 110 , an information integration module 120 , a prompt engineering 130 and a large language model 140 .

[0025] The input of the presentation parsing system 100 is one or more presentations, which can be collected in life or production process, and the format of the presentation can be PPT. The presentation includes multiple presentation pages, and multiple types of visual elements (shapes) are provided in each presentation page. Visual elements refer to various graphic and text components used to convey information in the presentation page of the presentation, such as text boxes, graphics, lines, pictures, tables, charts, etc.

[0026] like Figure 1 As shown, the feature extraction module 110 is used to extract the text information, position information and element type of each visual element in the document page of the presentation. The position information of the visual element refers to the specific placement of the visual element on the presentation page. It conveys the importance of the visual element in the document page and the relationship between each visual element. The element type refers to the type of each visual element. Text elements refer to the units that can be identified and analyzed in the visual elements that constitute the text, such as Chinese characters, letters, words, sentences, paragraphs, etc.

[0027] The feature extraction module 110 is also used to extract other feature information of each visual element based on the element type of each visual element. Visual elements of different visual types can provide different feature information. For example, visual elements with text symbols are used to provide text information. In addition to providing text information, visual elements with text symbols can also provide other feature information, such as multiple paragraphs and text style information, such as paragraph spacing, bold, italic, etc.

[0028] The feature extraction module 110 is also used to obtain layout information of each visual element according to the position information of each visual element. The layout information refers to the spatial arrangement and organization of the visual elements in the presentation page.

[0029] The information integration module 120 is used to combine the feature information and layout information of each visual element to form a description of the presentation page. The description format is a computer editable and searchable text format. It contains complete and structured context information in the presentation page.

[0030] The Prompt Project 130 is used to design special prompt words for presentation parsing tasks, guiding the large language model to perform specific tasks and generate specific outputs.

[0031] The large language model 140 is used to perform semantic analysis according to the description of each presentation page under the guidance of the prompt word Prompt, and output the analysis result of each presentation page according to a preset format, so as to finally obtain the analysis result of the presentation.

[0032] In summary, since the description includes not only the feature information of each visual element in the presentation page, but also the layout information of each visual element, using it as the input of the large language model for semantic parsing can improve the system's parsing effect on the presentation.

[0033] Figure 2 FIG. 1 shows a flow chart of a presentation parsing method based on feature extraction and a large language model provided in an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0034] Step S201, receiving a presentation, the presentation includes multiple presentation pages, and each presentation page includes multiple visual elements.

[0035] In one embodiment, the presentation includes multiple presentation pages, each of which includes multiple types of visual elements. The element types of the visual elements include at least text boxes, visual elements with text symbols, relational visual elements, tables, charts, and pictures.

[0036] Text box refers to the main text box, sub-text box, etc. of the presentation page, which is used to summarize the core content of the presentation page. It can be placed in a prominent position at the top, middle or bottom of the presentation page. Visual elements with text symbols are areas for adding text, such as basic graphics such as rectangles and circles, which can be freely placed anywhere on the presentation page. Relational shapes can include basic graphics such as arrows and lines, which are used to connect multiple visual elements into a whole to clearly display information. Tables are used to organize and display data sets, such as numbers, text lists, etc., and present information in the form of rows and columns. Charts are used to display data and trends, such as bar charts, pie charts, line charts, etc. Pictures can be bitmaps (such as JPEG, PNG) or vector graphics (such as SVG) to display products, scenes, people or other visual content.

[0037] For each visual element, not only the text information, such as description text, list items, etc., position information and element type are included. Other feature information is also included, such as relative position and alignment with other visual elements. This information is important for understanding word meaning, sentence structure, paragraph development, and the intention of the entire presentation page.

[0038] In some embodiments, the multiple types of visual elements also include speaker notes and annotations, which are additional text areas associated with the presentation page. It is used to record speech points or backup information and has text information.

[0039] Step S202, for each presentation page, call a library function in a preset presentation open source library to extract feature information of each visual element in the presentation page. The feature information of a single visual element includes at least text information, location information of the visual element, and element type of the visual element. Based on the location information, determine the layout information of each visual element, combine the feature information and layout information of each visual element, and form a description of the presentation page.

[0040] In one embodiment, the preset presentation open source library is a presentation software library whose source code is open to the public, allowing users to freely use, modify and distribute it.

[0041] Taking a presentation in PPT format as an example, the preset presentation open source library may be a Python library python-pptx for operating PPT files, referred to as pptx. pptx includes a variety of library functions to support a variety of functions, including but not limited to identifying presentation pages, identifying pictures, etc.

[0042] In one embodiment, before extracting the characteristic information of each visual element in the demonstration page, each combined visual element in the demonstration page is also disassembled to obtain multiple atomic visual elements as multiple visual elements. This is because the function in pptx usually takes the atomic visual element as the processing object. By disassembling the combined visual elements, the processing effect of the visual elements can be improved.

[0043] For each demonstration page, extract the feature information of each visual element in the demonstration page, including:

[0044] First, the first method in the preset presentation open source library is called to extract the text information, position information and element type of each visual element in the presentation page.

[0045] Each visual element included in the demo page has position attributes. The position attributes include the horizontal and vertical positions of the visual element on the demo page, as well as the width and height of the visual element.

[0046] Taking PPT as an example, for each presentation page, the first method in pptx can be called to extract the position information of each visual element in the presentation page. The extraction result can include the following information: The coordinates of the upper left corner of the bounding box to which each visual element belongs (x_min, y_min): The pixel coordinates of the upper left corner, usually with the upper left corner of the presentation page as the origin (0, 0), determine the horizontal position of the visual element on the presentation page: x_min and the vertical position: y_min. The coordinates of the lower right corner of the bounding box to which each visual element belongs (x_max, y_max): The pixel coordinates of the lower right corner determine the width and height of the visual element. Width: The width of the visual element, equal to x_max minus x_min. Height: The height of the visual element, equal to y_max minus y_min.

[0047] Secondly, the preset method in the preset presentation open source library is called to extract other feature information of each visual element.

[0048] In one embodiment, the visual element includes a text box and a visual element having a text symbol.

[0049] In another embodiment, the visual element includes a relational visual element. Extracting other characteristic information of each visual element includes: based on the element type of the relational visual element, calling a second method in a preset presentation open source library to extract text information from visual elements associated with the relational visual element.

[0050] In another embodiment, the visual element includes a chart. Extracting other feature information of each visual element includes: based on the element type of the chart, calling a third method in a preset presentation open source library to extract the type information of the chart and the context information in the chart, and classifying the information into the feature information. The type information of the chart includes the chart type, and the context information in the chart includes the chart title, the category of the chart data, and the value. The context information refers to the meaning connection and mutual dependence between the various parts of the visual element.

[0051] In another embodiment, the visual element includes a table. Extracting other feature information of each visual element includes: based on the element type of the table, calling the fourth method in the preset presentation open source library to extract context information and structural information of the table, and classifying them into the feature information; the context information of the table includes cell data in the table, and the structural information of the table includes row and column information of the cell data.

[0052] In another embodiment, the visual element includes a picture. Extracting other feature information of each visual element includes: based on the element type of the picture, extracting text information in the picture using optical character recognition technology.

[0053] In some embodiments, a preset method in a preset presentation open source library may also be called to extract text information of the speaker's notes and annotations in the presentation page.

[0054] Next, based on the position information of all visual elements in each demonstration page, the layout information of each visual element is determined.

[0055] Exemplarily, the layout information includes an arrangement order from top to bottom and / or an arrangement order from left to right.

[0056] Taking the determination of the arrangement order from top to bottom as an example, for each visual element, sort them in order from small to large according to the size of the upper left corner vertical coordinate y_min in the position information of the visual element. If the size of the upper left corner vertical coordinate y_min is the same, refer to the size of the upper left corner horizontal coordinate x_min in the position information of the visual element again. The smaller the x_min, the higher the ranking. It can be understood that the determination of the arrangement order from top to bottom also follows a similar order from left to right and from top to bottom, which will not be repeated.

[0057] Furthermore, the feature information and layout information of each visual element are combined to form a description of the demonstration page.

[0058] Exemplarily, feature information and layout information of each visual element may be combined into the description respectively.

[0059] The text information in the visual elements associated with the relational visual elements can also be combined. For example, for an arrow-type relational visual element, the text information connected to the "→" and the two ends (or multiple ends) of the arrow can be combined to form a triple (or multi-tuple) description, and multiple triple descriptions can be further combined into a description.

[0060] It can be seen that the description includes not only the feature information of each visual element in the demonstration page, but also the layout information of each visual element.

[0061] Furthermore, by providing the description as input to the large language model, the large language model can perform more efficient semantic parsing, thereby generating more accurate parsing results for the demonstration page.

[0062] Step S203, input the description and preset prompt words of each demonstration page into the large language model, so that it performs semantic analysis according to the description under the guidance of the preset prompt words, and outputs the analysis result of each demonstration page. The analysis results of all demonstration pages constitute the analysis result of the presentation.

[0063] In one embodiment, a special preset prompt word Prompt is designed to guide the large language model to perform a specific task and generate a specific output according to the description.

[0064] Taking PPT presentations as an example, special preset prompts can be designed based on the specific features of the visual elements in the PPT (such as location information and process information), and the large language model can be optimized to handle information confusion and inverted text in the description.

[0065] Optionally, the preset prompt words include at least one pre-labeled sample example, and the sample example is used to guide the large language model to understand and complete the semantic parsing task of the demonstration page, and output the parsing result of each demonstration page.

[0066] Exemplarily, the description and preset prompt words of each demonstration page are input into the large language model, and under the guidance of the preset prompt words, the large language model performs semantic analysis according to the description.

[0067] Therefore, the present application can obtain more accurate presentation page parsing results and ultimately obtain presentation parsing results by providing a large language model with a description of feature information and layout information of visual elements for semantic parsing, thereby improving the system's parsing effect on presentations.

[0068] It should be noted that although the operations of the method of the embodiment of the present application are described in a specific order in the above embodiment, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flow chart can change the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0069] In the following content, the above Figure 2 The implementation steps in are illustrated by way of example.

[0070] For example, Figure 3 A demonstration page provided by an embodiment of the present application is shown in FIG.

[0071] like Figure 3 As shown, taking the PPT format presentation as an example, Figure 3 The left picture shows a demo page to be parsed. Figure 3 The right figure marks the atomic visual elements (Shape) of the demonstration page, including: text boxes (Text_box) 301-304, visual elements with text symbols (Auto_shape) 305-307, and arrow-shaped relationship visual elements (Line) 308-309.

[0072] First, the first method of pptx is called to extract feature information of each visual element 301 - 309 in the presentation page.

[0073] For example, position information and element types of visual elements 301 - 309 .

[0074] For example, the text information extracted from text boxes 301-304 are respectively: "Substantive aspects of the research project", "First stage", "Second stage", and "Third stage".

[0075] For another example, the text information extracted from the visual elements 305-307 with text symbols are: "Application and Project Establishment", "Research and Management", and "Completion and Promotion".

[0076] Secondly, the preset method in the preset presentation open source library is called to extract other feature information of each visual element.

[0077] For example, based on the element type of the arrow-shaped relational visual element, the second method of pptx is called to extract the text information in the visual elements associated with the arrow-shaped relational visual element 308, which are respectively: the visual element 305 with text symbols "application and project establishment", and the visual element 306 with text symbols "research and management";

[0078] For another example, based on the element type of the arrow-shaped relational visual element, the second method of pptx is called to extract the text information in the visual elements associated with the arrow-shaped relational visual element 309, which are, from left to right: the visual element 306 with text symbols "research and management", and the visual element 307 with text symbols "completion and promotion".

[0079] Next, based on the position information of all visual elements in each demonstration page, the layout information of each visual element is obtained, including the arrangement order in the demonstration page:

[0080] like Figure 3 As shown, the order from top to bottom is: 301, 302, 303, 304, 305, 308, 306, 309, 307,

[0081] The order from left to right is: 301, 305, 302, 308, 306, 303, 309, 307, 304,

[0082] Furthermore, the characteristic information of the visual elements 301-309 and the arrangement order of the visual elements 301-309 in the demonstration page are combined to obtain a description of the demonstration page:

[0083] connections text:Connections:

[0084] Arrow_id 1: Shape#5 (application and project establishment) → Shape#6 (research and management)

[0085] Arrow_id 2: Shape#6 (research and management) → Shape#7 (completion and promotion)

[0086] vertical relations text: Vertical Relations (position information from top to bottom):

[0087] Shape#1 Text_box is at position 1 from top

[0088] Shape#2Text_box Stage 1: is at position 2 from top

[0089] Shape#3Text_box Stage 2: is at position 3 from top

[0090] Shape#4Text_box first stage: is atposition4from top

[0091] Shape#5Auto_shape application and project establishment: is at position 5from top

[0092] Shape#8Line No text is at position 6from top

[0093] Shape#6Auto_shape research and management: is at position 7from top

[0094] Shape#9Line No text is at position 8from top

[0095] Shape#7Auto_shape Conclusion and Extension: is at position 9 from top

[0096] horizontal relations text: Horizontal Relations (position information from left to right):

[0097] Shape#1 Text_box is at position 1 from left

[0098] Shape#5Auto_shape application and project establishment: is at position 2from left

[0099] Shape#2Text_box Stage 1: is at position 3 from left

[0100] Shape#8Line No text is at position 4from left

[0101] Shape#6Auto_shape research and management: is at position 5 from left

[0102] Shape#3Text_box Stage 2: is at position 6 from left

[0103] Shape#9Line No text is at position 7from left

[0104] Shape#7Auto_shape Conclusion and Extension: is at position 8 from left

[0105] Shape#4Text_box Stage 1: is at position 9 from left

[0106] Among them, taking vertical_relations_text: Vertical Relations (position information from top to bottom): Shape#1Text_box The substantive link of the research project: is at position 1from top as an example, it means that the element type of Shape#1 (visual element 301) is Text box, and the extracted feature information includes the text information "The substantive link of the research project", and the arrangement order of the visual element from top to bottom in the demonstration page: "is at position1from top".

[0107] For another example, Arrow_id 1: Shape#5 (application and project establishment) → Shape#6 (research and management) represents a triple relationship consisting of Arrow_id1 (visual element 308) and its associated Shape#5 (visual element 305) and Shape#6 (visual element 306).

[0108] For another example, Shape#9Line No text is at position 7from left means that the element type of Shape#9 (visual element 309) is Line, does not include text information (No text), and the arrangement order of the visual element from left to right in the demonstration page is: "is at position 7from left".

[0109] Finally, the specially designed prompt and description are input into the large language model, so that it can perform semantic interpretation according to the description under the guidance of the prompt, and obtain the parsing result of the demonstration page:

[0110] PPT description:

[0111] This PPT mainly introduces the substantive aspects of the research project. The specific aspects are as follows:

[0112] 1. The first stage: application and project establishment.

[0113] 2. The second stage: research and management.

[0114] 3. The third stage: conclusion and promotion.

[0115] Based on the method in the above embodiment, illustratively, Figure 4 A presentation parsing device based on feature extraction and a large language model provided by an embodiment of the present application is shown. Figure 4 As shown, the presentation parsing device 400 includes:

[0116] The receiving module 410 is used to receive a presentation document, wherein the presentation document includes a plurality of presentation pages, and each presentation page includes a plurality of visual elements.

[0117] The extraction module 420 is used to call the library function in the preset presentation open source library for each presentation page to extract the feature information of each visual element in the presentation page. The feature information of a single visual element at least includes text information, location information of the visual element, and element type of the visual element. Based on the location information, the layout information of each visual element is determined, and the feature information and layout information of each visual element are combined to form a description of the presentation page.

[0118] The parsing module 430 is used to input the description and preset prompt words of each demonstration page into the large language model, so that it performs semantic parsing according to the description under the guidance of the preset prompt words, and outputs the parsing result of each demonstration page. The parsing results of all demonstration pages constitute the parsing result of the presentation.

[0119] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device. The electronic device may include: at least one memory for storing a program; at least one processor for executing the program stored in the memory. Wherein, when the program stored in the memory is executed, the processor is used to execute the method described in the above embodiment. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a server, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, and a cellular phone, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR) device, a virtual reality (virtual reality, VR) device, an artificial intelligence (artificial intelligence, AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiment of the present application does not impose any special restrictions on the specific category of the electronic device.

[0120] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0121] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0122] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A presentation parsing method based on feature extraction and large language model, characterized in that: The method comprises: Receive a presentation; the presentation includes a plurality of presentation pages, each presentation page includes a plurality of visual elements; For each demonstration page, call a library function in a preset presentation open source library to extract feature information of each visual element in the demonstration page; the feature information of a single visual element at least includes text information, position information of the visual element, and element type of the visual element; based on the position information, determine the layout information of each visual element, combine the feature information and layout information of each visual element, and form a description of the demonstration page; The description and preset prompt words of each demonstration page are input into the large language model, so that it performs semantic analysis according to the description under the guidance of the preset prompt words, and outputs the analysis result of each demonstration page; the analysis results of all demonstration pages constitute the analysis result of the presentation.

2. The method according to claim 1, characterized in that Before extracting the feature information of each visual element in the demonstration page, the method further includes: Each combined visual element in the demonstration page is disassembled to obtain a plurality of atomic visual elements as the plurality of visual elements.

3. The method according to claim 1, characterized in that The step of extracting feature information of each visual element in the demonstration page includes: Calling the first method in the preset presentation open source library to extract text information, position information and element type of each visual element in the presentation page; Call the preset method in the preset presentation open source library to extract other feature information of each visual element.

4. The method according to claim 3, characterized in that The visual elements include text boxes and visual elements with text symbols.

5. The method according to claim 3, characterized in that: The visual elements include relational visual elements; The extracting of other feature information of each visual element includes: Based on the element type of the relational visual element, a second method in a preset presentation open source library is called to respectively extract text information in visual elements associated with the relational visual element.

6. The method according to claim 3, characterized in that The visual elements include charts; The extracting of other feature information of each visual element includes: Based on the element type of the chart, a third method in a preset presentation open source library is called to extract type information of the chart and context information in the chart, and include the feature information; the type information of the chart includes the chart type, and the context information in the chart includes the chart title, the category of the chart data, and the value.

7. The method according to claim 3, characterized in that The visual elements include tables; The extracting of other feature information of each visual element includes: Based on the element type of the table, the fourth method in the preset presentation open source library is called to extract the context information and structural information of the table, and include them in the feature information; the context information of the table includes the cell data in the table, and the structural information of the table includes the row and column information of the cell data.

8. The method according to claim 3, characterized in that The visual elements include pictures; The extracting of other feature information of each visual element includes: Based on the element type of the image, optical character recognition technology is used to extract text information from the image.

9. The method according to claim 1, characterized in that: The layout information includes an arrangement order from top to bottom and / or an arrangement order from left to right.

10. A presentation parsing device based on feature extraction and large language model, characterized in that: The device comprises: A receiving module, used for receiving a presentation document; the presentation document includes a plurality of presentation pages, each presentation page includes a plurality of visual elements; An extraction module is used to call a library function in a preset presentation open source library for each presentation page to extract feature information of each visual element in the presentation page; the feature information of a single visual element at least includes text information, position information of the visual element, and element type of the visual element; based on the position information, layout information of each visual element is determined, and the feature information and layout information of each visual element are combined to form a description of the presentation page; The parsing module is used to input the description and preset prompt words of each demonstration page into the large language model, so that it performs semantic parsing according to the description under the guidance of the preset prompt words, and outputs the parsing result of each demonstration page; the parsing results of all demonstration pages constitute the parsing result of the presentation.

Images