Demonstration document generation method and device, electronic equipment and storage medium

Abstract

The invention provides a presentation document generation method and device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and is suitable for the field of financial science and technology. The method comprises the following steps: performing information analysis on document generation indication information to obtain a document guide language and a document generation intention; selecting elements according to the document generation intention to obtain at least two document elements; generating a document outline structure according to each document element and the document guide language; wherein the document outline structure comprises a basic document page, and the basic document page is provided with a demonstration guide text; performing material selection on a pre-constructed structured database according to the demonstration guide text to obtain a target material and material attribute information; updating the basic document page according to the target material and the material attribute information to obtain a target document page; and replacing the basic document page in the document outline structure with the target document page to obtain a target demonstration document. According to the method, the presentation document generation accuracy can be improved, and personalized requirements of users are met.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, is applicable to the field of financial technology, and in particular to a method and device for generating a presentation document, an electronic device, and a storage medium. Background Art

[0002] A presentation document is a visual material used to display and convey information, the most common form of which is a slideshow. In the field of fintech, presentation documents can be used to provide customers with relevant business, such as insurance product introductions.

[0003] Related technologies usually generate new presentation documents based on existing presentation documents, or generate presentation texts based on limited document templates. However, the presentation documents generated by the above methods are relatively rigid in presentation order or presentation content, and cannot meet the personalized needs of users. Therefore, there is still a lot of room for improvement in the accuracy of presentation document generation. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a presentation document generation method and device, an electronic device, and a storage medium, which can improve the accuracy of presentation document generation and meet the personalized needs of users.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for generating a presentation document, the method comprising:

[0006] Get document generation instruction information;

[0007] Parsing the document generation instruction information to obtain document guide words and document generation intention;

[0008] Selecting elements according to the document generation intention to obtain at least two document elements;

[0009] Generating a document outline structure according to each of the document elements and the document guide words; wherein the document outline structure includes a basic document page, and the basic document page has a demonstration guide text;

[0010] Selecting materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information;

[0011] The basic document page is updated according to the target material and material attribute information to obtain a target document page;

[0012] The target document page is used to replace the basic document page in the document outline structure to obtain a target presentation document.

[0013] In some embodiments, before selecting materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information, the method further includes:

[0014] Constructing the structured database specifically includes:

[0015] Get the original image;

[0016] Extracting sub-pictures from the original picture to obtain sub-picture materials and sub-picture attribute information;

[0017] Structurally represent the sub-picture material and the sub-picture attribute information to obtain picture material structured data;

[0018] Extracting text from the original image to obtain text material and text attribute information;

[0019] Structurally represent the text material and the text attribute information to obtain text material structured data;

[0020] The picture material structured data and the text material structured data are stored to obtain the structured database.

[0021] In some embodiments, extracting sub-pictures from the original picture to obtain sub-picture materials and sub-picture attribute information includes:

[0022] Performing object detection on the original image to obtain a target object area;

[0023] Performing semantic segmentation on the target object area to obtain sub-image materials;

[0024] The sub-picture attribute information is generated according to the attribute characteristics of the sub-picture material.

[0025] In some embodiments, extracting text from the original image to obtain text material and text attribute information includes:

[0026] Performing text area detection on the original image to obtain text material and text layout; wherein the text layout is used to represent the paragraph to which the text material belongs;

[0027] Performing multi-scale feature extraction on the original image to obtain image features of multiple different scales;

[0028] Relationship recognition is performed on each of the image features and the text layout to obtain a text relationship; wherein the text relationship is used to characterize the relationship between at least two text segments;

[0029] The text attribute information is generated according to the text relationship.

[0030] In some embodiments, the step of identifying the relationship between each of the image features and the text layout to obtain the text relationship includes:

[0031] Performing feature representation on the text layout to obtain text layout features;

[0032] Performing feature fusion on each of the image features to obtain image fusion features;

[0033] Perform feature multiplication processing on the text layout feature and the image fusion feature through a preset relationship weight matrix to obtain a picture-text relationship matrix;

[0034] Perform feature multiplication processing according to the text layout feature, the image fusion feature and the image-text relationship matrix to obtain an image attention weight vector;

[0035] Perform feature multiplication processing on the image attention weight vector and the image fusion feature to obtain an image attention feature;

[0036] Perform feature multiplication processing according to the text layout feature, the image fusion feature and the transposed matrix of the image-text relationship matrix to obtain a text attention weight vector;

[0037] Perform feature multiplication processing according to the text attention weight vector and the text layout feature to obtain a text attention feature;

[0038] Relation encoding is performed on the image attention feature and the text attention feature to obtain the text relationship.

[0039] In some embodiments, the structured database includes at least two of the picture material structured data and the text material structured data corresponding to the original picture;

[0040] The method of searching a structured database for materials according to the demonstration guide text to obtain target materials and material attribute information includes:

[0041] Filter each of the original images according to the demonstration guide text to obtain a target image;

[0042] Determining at least one of the picture material structured data and the text material structured data corresponding to the target picture as target material structured data;

[0043] The target material and material attribute information are extracted from the target material structured data.

[0044] In some embodiments, the step of screening each of the original images according to the demonstration guide text to obtain the target image includes:

[0045] Performing semantic recognition on each of the original images to obtain semantic information of the image;

[0046] Performing similarity calculation based on the demonstration guide text and the image semantic information to obtain semantic similarity;

[0047] The target image is selected from each of the original images according to the semantic similarity.

[0048] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a presentation document generating device, the device comprising:

[0049] An information acquisition module, used to acquire document generation instruction information;

[0050] An information parsing module, used to parse the document generation instruction information to obtain document guide words and document generation intention;

[0051] An element selection module, used to select elements according to the document generation intention to obtain at least two document elements;

[0052] An outline generation module, used to generate a document outline structure according to each of the document elements and the document guide words; wherein the document outline structure includes a basic document page, and the basic document page has a demonstration guide text;

[0053] A material selection module is used to select materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information;

[0054] A document page updating module, used to update the basic document page according to the target material and material attribute information to obtain a target document page;

[0055] The document determination module is used to replace the basic document page in the document outline structure with the target document page to obtain a target presentation document.

[0056] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the presentation document generation method described in the first aspect when executing the computer program.

[0057] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating a presentation document described in the first aspect is implemented.

[0058] The method and device for generating a presentation document, an electronic device, and a storage medium proposed in the present application first obtain document generation indication information, and obtain document guide words and document generation intention after parsing. The document generation intention is used to characterize the user's intention to generate a presentation document, so that the document elements selected according to the document generation intention meet the user's intention. The document guide words are used to characterize the guide text for generating a presentation document, so that the document outline structure generated by the document guide words and the document elements not only meets the user's intention, but also meets the user's personalized needs for the document content. In order to further enrich the document content, the target material and material attribute information are also selected based on the demonstration guide text in the basic document page. The target material refers to the content that can be typeset on the document page, and the material attribute information is used to characterize the attribute information of the target material itself, and provide a reference for the typesetting of the target material on the basic document page, thereby obtaining the target document page. Finally, the target document page is replaced with the basic document page to obtain the target presentation document. In summary, the embodiments of the present application can improve the accuracy of presentation document generation and meet the personalized needs of users.

[0059] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flowchart of a method for generating a presentation document provided in an embodiment of the present application;

[0061] Figure 2 is a flowchart of a method for generating a presentation document provided by another embodiment of the present application;

[0062] Figure 3 yes Figure 2 Flowchart of step 202 in FIG.

[0063] Figure 4 yes Figure 2 Flowchart of step 204 in;

[0064] Figure 5 yes Figure 4 Flow chart of step 403 in;

[0065] Figure 6 yes Figure 1 Flowchart of step 105 in;

[0066] Figure 7 It is a module structure block diagram of the presentation document generating device provided in an embodiment of the present application;

[0067] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] First, some nouns involved in this application are analyzed:

[0072] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese, English, etc.). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information image processing, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0074] The embodiments of the present application provide a method for generating a presentation document, a device for generating a presentation document, an electronic device, and a computer-readable storage medium, that is, a technology for adaptively and flexibly generating a target presentation document based on document generation instruction information. The embodiments of the present application aim to efficiently understand and process input document generation instruction information by combining natural language processing technology (NLP) and a pre-set rule engine. A speech outline is constructed based on specific document generation intentions and document guides in the document generation instruction information to ensure that the results meet user expectations and present good logic and consistency.

[0075] The method for generating a presentation document provided in the embodiment of the present application can be applied to a terminal and a server, and can also be software running on the server. The server can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the method for generating a presentation document, etc., but is not limited to the above forms.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: server computers, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0077] The embodiments of the present application provide a presentation document generation method and a presentation document generation device, an electronic device, and a computer-readable storage medium, which are specifically illustrated by the following embodiments. First, the presentation document generation method in the embodiments of the present application is described.

[0078] It should be noted that in each specific implementation method of the present application, when it comes to processing data related to the user's identity or characteristics such as the user's behavioral data, the user's permission or consent will be obtained first, and the collection, use and processing of such data will comply with relevant laws, regulations and standards.

[0079] Reference Figure 1 , Figure 1 It is an optional flowchart of the presentation document generation method provided in an embodiment of the present application, which may include but is not limited to steps 101 to 107.

[0080] Step 101, obtaining document generation instruction information;

[0081] Step 102, parsing the document generation instruction information to obtain the document guide and document generation intention;

[0082] Step 103, selecting elements according to the document generation intention to obtain at least two document elements;

[0083] Step 104, generating a document outline structure according to each document element and the document guide; wherein the document outline structure includes a basic document page, and the basic document page has a demonstration guide text;

[0084] Step 105, selecting materials from the pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information;

[0085] Step 106, updating the basic document page according to the target material and material attribute information to obtain the target document page;

[0086] Step 107, replacing the basic document page in the document outline structure with the target document page to obtain the target presentation document.

[0087] In the steps 101 to 107 shown in the embodiment of the present application, the document generation indication information is first obtained, and the document guide and the document generation intention are obtained after parsing. The document generation intention is used to characterize the user's intention to generate a presentation document, so that the document elements selected according to the document generation intention meet the user's intention. The document guide is used to characterize the guide text for generating a presentation document, so that the document outline structure generated by the document guide and the document elements not only meets the user's intention, but also meets the user's personalized needs for the document content. In order to further enrich the document content, the target material and material attribute information are also selected based on the demonstration guide text in the basic document page. The target material refers to the content that can be typeset on the document page, and the material attribute information is used to characterize the attribute information of the target material itself, and provide a reference for the typesetting of the target material on the basic document page, thereby obtaining the target document page. Finally, the target document page is replaced with the basic document page to obtain the target presentation document. In summary, the embodiment of the present application can improve the accuracy of presentation document generation and meet the personalized needs of users.

[0088] In step 101 of some embodiments, document generation instruction information is obtained. The document generation instruction information refers to information requesting the generation of a demonstration document. The document generation instruction information includes information related to the generation of the document. For example, the text generation instruction information is "For the auto insurance product, please generate a demonstration document for me to introduce the auto insurance product."

[0089] In one embodiment, the method of obtaining the document generation instruction information includes but is not limited to the following situations:

[0090] (1) Natural language input: Users can express their needs by entering text descriptions, for example, entering complete sentences or phrases in a chat interface.

[0091] (2) Forms or selection boxes: Pre-designed forms or selection boxes allow users to select or fill in specific options. This method is suitable for structured data collection.

[0092] (3) Speech recognition: Using the speech input function, users can generate document generation instructions by speaking. This requires corresponding speech recognition technology support.

[0093] (4) Graphical User Interface (GUI) Interaction: Through graphical buttons and menus, users can click on corresponding operations to generate document generation instructions.

[0094] In step 102 of some embodiments, the document generation instruction information is parsed to obtain the document guide and document generation intention. Natural language processing technology can be used for information parsing. For example, for the above example text generation instruction information, "For auto insurance products, please generate a demonstration document for me to introduce auto insurance products", after parsing with the large language model, the document guide can be obtained as "auto insurance products or auto insurance product introduction", and the document generation intention is "for product promotion or product marketing".

[0095] Large Language Model (LLM) refers to an artificial intelligence system that processes and generates natural language text based on deep learning technology, especially neural network models. Large language models are usually trained on large amounts of text data to learn the structure, grammar, semantics, and contextual relationships of the language, so that they can perform a variety of language tasks. The following are some of the main features of large language models: Large scale: Large language models usually contain hundreds of millions to hundreds of billions of parameters, enabling them to capture complex language features. Large training data: Use large-scale text data for training, covering a variety of topics and styles to improve the model's language understanding and generation capabilities. Task diversity: Can perform a variety of natural language processing tasks, such as text generation, answering questions, translation, summarization, sentiment analysis, etc. Context understanding: Ability to understand and generate coherent text that conforms to the context, process longer text inputs, and help maintain the coherence of the conversation. Self-learning: Through pre-training and fine-tuning, it can be optimized on specific tasks or fields to improve performance.

[0096] In step 103 of some embodiments, elements are selected according to the document generation intent to obtain at least two document elements. Document elements refer to the components required to compose a presentation document. The element selection rules can be configured in advance to facilitate element selection through the element selection rules and the document generation intent. The element selection rules include candidate intents and at least two candidate elements corresponding to the candidate intents. Specifically, the candidate intent that is the same as the document generation intent is used as the target intent, and at least two document elements are obtained according to each candidate element corresponding to the target intent. For example, using a pre-set element selection rule, if the document generation intent is "generate an insurance business development plan", the selected document elements include a cover, a self-introduction, macro data, insurance concepts, customized demand analysis, plan recommendations, value-added service introductions, etc.

[0097] In step 104 of some embodiments, a document outline structure is generated based on various document elements and document guides. The document outline structure includes at least two basic document pages. The basic document pages include a title page, an introduction page, a chapter homepage, at least two chapter subpages, a conclusion page, etc. Each basic document page has a demonstration guide text. For example, based on the above example, the document generation intention is "Generate a business development plan", then the demonstration guide text of the title page is "Please provide title text / picture related to insurance business development", the demonstration guide document of the introduction page is "Please describe the background information related to insurance business development, document purpose, and main content overview", etc., the chapter homepage is "Please describe the main content of this chapter, and please provide chapter text / picture related to insurance business development", etc.

[0098] In step 105 of some embodiments, material selection is performed on a pre-built structured database according to the demonstration guide text to obtain target material and material attribute information. A structured database refers to a database that stores structured data. Structured data refers to data including the material (text / picture) itself and material attribute characteristics. Structured data includes picture material structured data and text material structured data. Picture material structured data includes pictures and attribute characteristics including picture position coordinates, size, etc. Text material structured data includes text and attribute characteristics including text semantics and text relations, etc.

[0099] In one embodiment, before step 105, refer to Figure 2 The method for generating a demonstration text provided in this embodiment may further include: constructing a structured database, specifically including:

[0100] Step 201, obtaining the original image;

[0101] Step 202, extract sub-pictures from the original picture to obtain sub-picture materials and sub-picture attribute information;

[0102] Step 203, structurally representing the sub-picture material and sub-picture attribute information to obtain picture material structured data;

[0103] Step 204, extracting text from the original image to obtain text material and text attribute information;

[0104] Step 205, structurally representing the text material and text attribute information to obtain text material structured data;

[0105] Step 206: Store the picture material structured data and the text material structured data to obtain a structured database.

[0106] In step 201, the original image refers to the image itself and does not carry additional information. In actual production and life, insurance companies have accumulated a large amount of image resources in material production, but due to problems with storage methods or production methods, many resources exist only in the form of images, and the source files of the images are missing. For example, PPT will contain information such as image elements, image formats, image relationships, text content, text levels, and text relationships. Pure images cannot be reused when making new materials due to the lack of such information, resulting in a waste of data. Even if they can be reused, they are mostly based on pure OCR recognition and keyword matching, and then the whole picture is spliced ​​and inserted. For the final material, there is a lack of more fine-grained logical processing and visual processing capabilities. Therefore, it is necessary to introduce more powerful image understanding capabilities to be able to perform a deeper modal deconstruction on pure images, so that the data obtained can generate greater value at the practical application level. Therefore, one of the purposes of this embodiment is to mine information for images and improve the information reuse rate.

[0107] In step 202, sub-pictures are extracted from the original picture to obtain sub-picture materials and sub-picture attribute information. The sub-picture attribute information is used to characterize the attribute characteristics of the sub-picture materials. For example, the sub-picture attribute information includes the position coordinate information and size information of the sub-picture materials.

[0108] In one embodiment, referring to Figure 3 , step 202 may include:

[0109] Step 301, performing object detection on the original image to obtain the target object area;

[0110] Step 302, semantic segmentation is performed on the target object area to obtain sub-image materials;

[0111] Step 303, generating sub-picture attribute information according to the attribute characteristics of the sub-picture material.

[0112] In step 301, the deep learning model YOLO is used to identify and locate objects contained in the original image to obtain the target object area.

[0113] In step 302, the semantic segmentation model Mask R-CNN is used to accurately mark and segment each object and its corresponding region in the target object region to obtain sub-image materials and attribute features of the sub-image materials.

[0114] In step 303, sub-picture attribute information is generated according to the attribute characteristics of the sub-picture material, such as position coordinates and size.

[0115] The benefit of the above-mentioned embodiment of step 301 to step 303 is that, on the basis of achieving the extraction of sub-picture material and sub-picture attribute information from the original picture, the extraction accuracy is improved.

[0116] In another embodiment, the picture information prompt text may be pre-set, and the picture information extraction guide text and the original picture may be extracted through a pre-trained large language model to obtain sub-picture material and sub-picture attribute information.

[0117] In step 203, the sub-picture material and the sub-picture attribute information are structured in a data table form or a database form to obtain picture material structured data;

[0118] In step 204, text extraction is performed on the original image to obtain text material and text attribute information. The text attribute information is used to characterize the attribute characteristics of the text material. For example, the text attribute information includes the text semantics and text relationship of the text material. The text relationship is used to characterize the relationship between at least two text segments.

[0119] In one embodiment, referring to Figure 4 , step 204 may include:

[0120] Step 401, performing text area detection on the original image to obtain text material and text layout; wherein the text layout is used to represent the paragraph to which the text material belongs;

[0121] Step 402, performing multi-scale feature extraction on the original image to obtain image features of multiple different scales;

[0122] Step 403, performing relationship recognition on each image feature and text layout to obtain text relationship;

[0123] Step 404: Generate text attribute information based on the text relationship.

[0124] In step 401, a BROS model can be used as a text region detection model, which is a region-based text detection and recognition model. The BROS model can detect text regions in the original image to obtain text materials, and assign them to corresponding text lines and paragraphs to obtain text layout.

[0125] In step 402, ConvNeXt and multi-scale FPN can be used as the backbone network of the visual module to extract image features of different scales. ConvNeXt is a convolutional neural network structure that performs convolution operations in multiple dimensions to improve the expressiveness of the model. Multi-scale FPN is a feature pyramid network that can extract features at different scales to accommodate texts of different sizes.

[0126] In step 403 , the textual relations include the relations between two text segments, the relations between a plurality of text segment pairs, and the relations between three text segments.

[0127] In one embodiment, referring to Figure 5 , step 403 may include:

[0128] Step 501, feature representation of text layout to obtain text layout features;

[0129] Step 502, performing feature fusion on each image feature to obtain an image fusion feature;

[0130] Step 503, performing feature multiplication processing on the text layout features and the image fusion features through a preset relationship weight matrix to obtain a picture-text relationship matrix;

[0131] Step 504, performing feature multiplication processing according to the text layout feature, the image fusion feature and the image-text relationship matrix to obtain the image attention weight vector;

[0132] Step 505, performing feature multiplication processing according to the image attention weight vector and the image fusion feature to obtain the image attention feature;

[0133] Step 506, performing feature multiplication processing according to the text layout feature, the image fusion feature and the transposed matrix of the image-text relationship matrix to obtain a text attention weight vector;

[0134] Step 507, performing feature multiplication processing according to the text attention weight vector and the text layout feature to obtain the text attention feature;

[0135] Step 508, performing relation encoding on the image attention features and the text attention features to obtain the textual relationship.

[0136] In step 501, the text layout may be characterized by a feature vector embedding model (such as a BERT model) to obtain text layout features.

[0137] In step 502, each image feature may be spliced ​​to obtain an image splicing feature, and the image splicing feature may be determined as an image fusion feature. After obtaining the image splicing feature, the image splicing feature may be pooled to obtain an image fusion feature of the same dimension as the text layout feature.

[0138] In step 503, the relationship weight matrix is ​​a weight matrix for extracting the relationship between two features. Specifically, step 503 includes: multiplying the transpose of the text layout feature with the relationship weight matrix to obtain a first multiplication matrix; multiplying the first multiplication matrix with the image fusion feature to obtain a second multiplication matrix; performing nonlinear activation on the second multiplication matrix to obtain a picture-text relationship matrix. The nonlinear activation can select a hyperbolic tangent function, i.e., a Tanh function.

[0139] In step 504, the image attention weight vector refers to a vector composed of attention weights corresponding to elements of the image fusion feature. Specifically, step 504 includes: multiplying a preset image weight matrix with the image fusion feature to obtain a first image multiplication matrix; multiplying a text weight matrix with a text layout feature to obtain a text influence image matrix; multiplying the text influence image matrix with the image text relationship matrix to obtain a second image multiplication matrix; adding the first image multiplication matrix and the second image multiplication matrix to obtain an image addition matrix; performing nonlinear activation on the image addition matrix to obtain an image attention weight matrix; multiplying the transpose of the preset image weight parameter vector with the image attention weight matrix to obtain an initial image attention weight vector; performing nonlinear mapping on the initial image attention weight vector to obtain an image attention weight vector. A softmax function can be used for nonlinear mapping.

[0140] In step 505, feature multiplication is performed on the picture attention weight vector and the picture fusion feature to obtain the picture attention feature. The elements of the picture attention weight vector are multiplied with the elements of the picture fusion feature to obtain the picture attention feature elements; the elements of the picture fusion feature are replaced with the picture attention feature elements to obtain the picture attention feature.

[0141] In step 506, the text attention weight vector refers to a vector composed of attention weights corresponding to the elements of the text layout feature. Specifically, step 506 includes: multiplying the preset text weight matrix with the text layout feature to obtain a first text multiplication matrix; multiplying the image weight matrix with the image fusion feature to obtain the image influence text matrix; multiplying the image influence text matrix and the image text relationship matrix according to the transpose to obtain the second text multiplication matrix; adding the first text multiplication matrix and the second text multiplication matrix to obtain the text addition matrix; performing nonlinear activation on the text addition matrix to obtain the text attention weight matrix; multiplying the transpose of the preset text weight parameter vector with the text attention weight matrix to obtain the initial text attention weight vector; performing nonlinear mapping on the initial text attention weight vector to obtain the text attention weight vector. The nonlinear mapping can use a softmax function.

[0142] In step 507, feature multiplication processing is performed on the text attention weight vector and the text layout feature to obtain a text attention feature. The elements of the text attention weight vector are multiplied with the elements of the text layout feature to obtain a text attention feature element; the elements of the text layout feature are replaced with the text attention feature elements to obtain a text attention feature.

[0143] In step 508, the image attention feature and the text attention feature are relationally encoded to obtain a text relationship. Specifically, step 508 may include: adding the image attention feature and the text attention feature to obtain an image-text addition feature; multiplying a preset fusion weight parameter matrix with the image-text addition feature to obtain a fused image-text vector; multiplying a preset mapping weight parameter matrix with the fused image-text vector to obtain a target fused image-text feature; and performing nonlinear mapping on the target fused image-text feature to obtain a text relationship.

[0144] The embodiment of the above steps 501 to 508 aims to improve the accuracy of textual relations.

[0145] In step 205, the text material and text attribute information are structured in a data table form or a database form to obtain text material structured data.

[0146] In step 206, an initial database may be created first, and then the picture material structured data and the text material structured data may be added to the initial database to obtain a structured database.

[0147] The benefit of the above-mentioned embodiment of step 201 to step 206 is that a structured database is created based on the original image, which is convenient for improving the information reuse rate when generating documents.

[0148] In one embodiment, the structured database includes at least two picture material structured data and text material structured data corresponding to the original pictures. Figure 6 , step 105 may include:

[0149] Step 601, screening each original picture according to the demonstration guide text to obtain a target picture;

[0150] Step 602, determining at least one of the picture material structured data and the text material structured data corresponding to the target picture as the target material structured data;

[0151] Step 603: extract the target material and material attribute information from the target material structured data.

[0152] In step 601, the target image refers to an original image that meets the conditions of relevance to the demonstration guide text. The relevance includes semantic similarity. In one embodiment, step 601 may include: performing semantic recognition on each original image to obtain image semantic information; performing similarity calculation based on the demonstration guide text and the image semantic information to obtain semantic similarity; and selecting a target image from each original image based on the semantic similarity. In this way, the accuracy of determining the target image is improved.

[0153] In one embodiment, the above-mentioned step of performing semantic recognition on each original image to obtain image semantic information may include: performing image description on the original image through a large language model to obtain image description text; performing semantic information extraction on the image description text through a semantic recognition model to obtain image semantic information.

[0154] In one embodiment, the above-mentioned step of calculating similarity based on the demonstration guidance text and the picture semantic information to obtain semantic similarity may include: encoding the demonstration guidance text through an encoder to obtain a demonstration guidance text vector; encoding the picture semantic information through an encoder to obtain a picture semantic information vector; performing vector similarity calculation on the demonstration guidance text vector and the picture semantic information vector to obtain semantic similarity.

[0155] In one embodiment, the above-mentioned step of determining the original picture whose semantic similarity meets the predetermined conditions as the target picture may include: determining the original picture with the greatest semantic similarity as the target picture, or determining the original picture with a semantic similarity greater than a similarity threshold as the target picture, or sorting the original pictures from high to low according to the semantic similarity, and determining the original picture with a preset order in front as the target picture.

[0156] In step 602, the picture material structured data corresponding to the target picture can be determined as the target material structured data, the text material structured data corresponding to the target picture can be determined as the target material structured data, or the picture material structured data and text material structured data corresponding to the target picture can be determined as the target material structured data.

[0157] In step 603, if the target material structured data includes picture material structured data, the target material includes sub-picture material, and the target material attribute information includes sub-picture attribute information. If the target material structured data includes text material structured data, the target material includes text material, and the target material attribute information includes text attribute information.

[0158] The benefit of the embodiment of the above steps 601 to 603 is that the original image is utilized, but the presentation document can be generated in a more fine-grained manner according to the information in the original image, thereby improving the information reuse rate.

[0159] In step 106 of some embodiments, the base document page is updated according to the target material and material attribute information to obtain the target document page. Specifically, the insertion position is identified according to the material attribute information and the base document page to obtain the target insertion position, and the target material is inserted into the base document page according to the target insertion position to obtain the target document page. The target document page can also be updated by improving visual technology, including but not limited to color conversion, filter application and adjustment, quality enhancement operations, etc., and using the neural network architecture GAN to complete more complex style transfer tasks.

[0160] Finally, in step 107, the target document page is used to replace the basic document page in the document outline structure to obtain the target presentation document.

[0161] In one example, slide editing software can be used to automatically create new slides and insert stylized images and texts into appropriate locations in accordance with the outline organization structure. After replacing the basic document pages, the target presentation document is exported to PPT format.

[0162] In summary of the above embodiments, the present application can at least achieve the following effects: 1. It can automatically process the original image. The user only needs to input the instruction information to quickly generate the corresponding document outline through sub-image and text extraction, layout analysis and other functions, which greatly saves time and human resources. 2. The image information and text information are integrated and encoded and decoded to achieve the establishment and understanding of the relationship between them at different levels. This interactive method helps to fully grasp the connection between multimodal content and lays the foundation for subsequent steps such as generating outline structure and target document page. 3. The sub-image material and its related information are stored in the form of a data table or database, making subsequent retrieval and matching more efficient and fast. Each sub-image has rich feature attribute records, which can easily call logically consistent component information during the assembly process for reliable and effective document organization. 4. Combined with natural language processing technology (NLP) and a pre-set rule engine, it can efficiently understand and process the instruction information entered by the user. Build a document outline according to specific goals and requirements to ensure that the results meet user expectations and present good logic and consistency.

[0163] See also Figure 7 The embodiment of the present application also provides a presentation document generation device, which can implement the above presentation document generation method. Figure 7This is a module structure block diagram of the presentation document generation device provided in an embodiment of the present application, the device comprising: an information acquisition module 701, an information analysis module 702, an element selection module 703, an outline generation module 704, a material selection module 705, a document page update module 706 and a document determination module 707. Among them, the information acquisition module 701 is used to obtain document generation indication information; the information parsing module 702 is used to parse the document generation indication information to obtain document guide words and document generation intentions; the element selection module 703 is used to select elements according to the document generation intentions to obtain at least two document elements; the outline generation module 704 is used to generate a document outline structure according to each document element and document guide words; wherein the document outline structure includes a basic document page, and the basic document page has a demonstration guide text; the material selection module 705 is used to select materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information; the document page update module 706 is used to update the basic document page according to the target material and material attribute information to obtain the target document page; the document determination module 707 is used to replace the basic document page in the document outline structure with the target document page to obtain the target demonstration document.

[0164] In one embodiment, the presentation document generating device further comprises a database building module for building a structured database. The specific building process can refer to the above method embodiment.

[0165] It should be noted that the specific implementation of the presentation document generating device is substantially the same as the specific implementation of the above-mentioned presentation document generating method, and will not be described in detail herein.

[0166] The embodiment of the present application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned presentation document generation method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0167] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0168] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0169] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the presentation document generation method of the embodiment of this application;

[0170] Input / output interface 803, used to implement information input and output;

[0171] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0172] A bus 805 that transmits information between the various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0173] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0174] An embodiment of the present application also provides a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned presentation document generation method.

[0175] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0177] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0178] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0180] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0181] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0182] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0186] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A method for generating a presentation document, characterized in that: The method comprises: Get document generation instruction information; Parsing the document generation instruction information to obtain document guide words and document generation intention; Selecting elements according to the document generation intention to obtain at least two document elements; Generating a document outline structure according to each of the document elements and the document guide words; wherein the document outline structure includes a basic document page, and the basic document page has a demonstration guide text; Selecting materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information; The basic document page is updated according to the target material and material attribute information to obtain a target document page; The target document page is used to replace the basic document page in the document outline structure to obtain a target presentation document.

2. The method according to claim 1, characterized in that Before selecting materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information, the method further includes: Constructing the structured database specifically includes: Get the original image; Extracting sub-pictures from the original picture to obtain sub-picture materials and sub-picture attribute information; Structurally represent the sub-picture material and the sub-picture attribute information to obtain picture material structured data; Extracting text from the original image to obtain text material and text attribute information; Structurally represent the text material and the text attribute information to obtain text material structured data; The picture material structured data and the text material structured data are stored to obtain the structured database.

3. The method according to claim 2, characterized in that The extracting sub-pictures from the original pictures to obtain sub-picture materials and sub-picture attribute information includes: Performing object detection on the original image to obtain a target object area; Performing semantic segmentation on the target object area to obtain sub-image materials; The sub-picture attribute information is generated according to the attribute characteristics of the sub-picture material.

4. The method according to claim 2, characterized in that: The extracting text from the original image to obtain text material and text attribute information includes: Performing text area detection on the original image to obtain text material and text layout; wherein the text layout is used to represent the paragraph to which the text material belongs; Performing multi-scale feature extraction on the original image to obtain image features of multiple different scales; Relationship recognition is performed on each of the image features and the text layout to obtain a text relationship; wherein the text relationship is used to characterize the relationship between at least two text segments; The text attribute information is generated according to the text relationship.

5. The method according to claim 4, characterized in that The step of identifying the relationship between each of the image features and the text layout to obtain the text relationship includes: Performing feature representation on the text layout to obtain text layout features; Performing feature fusion on each of the image features to obtain image fusion features; Perform feature multiplication processing on the text layout feature and the image fusion feature through a preset relationship weight matrix to obtain a picture-text relationship matrix; Perform feature multiplication processing according to the text layout feature, the image fusion feature and the image-text relationship matrix to obtain an image attention weight vector; Perform feature multiplication processing on the image attention weight vector and the image fusion feature to obtain an image attention feature; Perform feature multiplication processing according to the text layout feature, the image fusion feature and the transposed matrix of the image-text relationship matrix to obtain a text attention weight vector; Perform feature multiplication processing according to the text attention weight vector and the text layout feature to obtain a text attention feature; Relation encoding is performed on the image attention feature and the text attention feature to obtain the text relationship.

6. The method according to any one of claims 2 to 5, characterized in that: The structured database includes the picture material structured data and the text material structured data corresponding to at least two of the original pictures; The method of searching a structured database for materials according to the demonstration guide text to obtain target materials and material attribute information includes: Filter each of the original images according to the demonstration guide text to obtain a target image; Determining at least one of the picture material structured data and the text material structured data corresponding to the target picture as target material structured data; The target material and material attribute information are extracted from the target material structured data.

7. The method according to claim 6, characterized in that The step of screening each of the original images according to the demonstration guide text to obtain a target image includes: Performing semantic recognition on each of the original images to obtain semantic information of the image; Performing similarity calculation based on the demonstration guide text and the image semantic information to obtain semantic similarity; The target image is selected from each of the original images according to the semantic similarity.

8. A presentation document generating device, characterized in that: The device comprises: An information acquisition module, used to acquire document generation instruction information; An information parsing module, used to parse the document generation instruction information to obtain document guide words and document generation intention; An element selection module, used to select elements according to the document generation intention to obtain at least two document elements; An outline generation module, used to generate a document outline structure according to each of the document elements and the document guide words; wherein the document outline structure includes a basic document page, and the basic document page has a demonstration guide text; A material selection module is used to select materials from a pre-built structured database according to the demonstration guide text to obtain target materials and material attribute information; A document page updating module, used to update the basic document page according to the target material and material attribute information to obtain a target document page; The document determination module is used to replace the basic document page in the document outline structure with the target document page to obtain a target presentation document.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images