Video generation method and device, equipment and medium

Abstract

The invention belongs to the field of artificial intelligence, and relates to a video generation method comprising the following steps: obtaining a demonstration file; performing keyword extraction processing on the demonstration file to obtain a plurality of target keywords of the demonstration file; obtaining target material data of the demonstration file based on the plurality of target keywords; and fusing the demonstration file and the target material data to generate a video corresponding to the demonstration file. The invention further provides a video generation device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the demonstration file, the demonstration file, the target material data and the video generated by fusing the demonstration file and the target material data can be stored in the block chain. The quality of the video generated by the demonstration file can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a video generation method, device, equipment and medium. Background Art

[0002] With the increasing demand for digital presentations, people hope to convert static presentation files into more dynamic and attractive video formats so that they can be shared and disseminated in various occasions. For example, in the medical field, people hope to generate disease prevention promotion videos based on disease prevention promotion presentation files; in the financial field, people hope to generate investment strategy report videos based on investment strategy presentation files. However, the current web-based presentation file generation video technology has some problems and cannot generate high-quality videos corresponding to presentation files.

[0003] The existing technology mainly relies on parsing the text and image elements in the presentation file and converting them into video format. However, this method has limited ability to parse and retain special elements in the presentation file, such as animation effects, embedded audio and video, etc., and it is difficult to accurately extract these special elements and convert them into video format. Therefore, the animation effects in the presentation file may not be fully presented in the generated video, which limits the visual expressiveness of the video. The generated video cannot present rich content and visual effects and has low quality. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video generation method, device, equipment and medium to solve the problem of low video quality corresponding to the existing generated presentation files.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a video generation method, which adopts the following technical solutions:

[0006] Acquire a demonstration file; perform keyword extraction processing on the demonstration file to obtain multiple target keywords of the demonstration file; based on the multiple target keywords, acquire target material data of the demonstration file; merge the demonstration file and the target material data to generate a video corresponding to the demonstration file.

[0007] Further, the step of performing keyword extraction processing on the demonstration file to obtain multiple target keywords of the demonstration file specifically includes:

[0008] The demonstration file is parsed to obtain structural data and text data of the demonstration file; the text data is subjected to keyword extraction processing to obtain multiple candidate keywords of the demonstration file; based on the structural data, the multiple candidate keywords are screened to obtain multiple target keywords of the demonstration file.

[0009] Furthermore, before the step of acquiring target material data of the presentation file based on the multiple target keywords, the method specifically includes:

[0010] Acquire reference material data, perform feature extraction on the reference material data, and obtain feature information of the reference material; classify the reference material data using a clustering algorithm based on the feature information to obtain material data sets of different categories; construct index information based on the material data sets of each category; and construct a material library based on the reference material data and the index information.

[0011] Furthermore, after the step of constructing a material library based on the reference material data and the index information, the method further includes:

[0012] According to the preset time interval, crawl the target network data from the network; based on the preset material crawling rules, parse and extract the crawled network data to obtain the original material data; determine whether the extracted original material data exists in the material library. If not, add the original material data to the material library.

[0013] Furthermore, based on the multiple target keywords, the step of obtaining the target material data of the presentation file specifically includes:

[0014] Taking multiple target keywords as query information, obtaining query results corresponding to the query information from the material library; determining whether the query results contain reference material data; if the query results contain reference material data, obtaining multiple material feature information of the reference material data; extracting key feature information of multiple target keywords, performing information fusion on the key feature information, and obtaining target key feature information; calculating the similarity between the target key feature information and multiple material key feature information, and obtaining a similarity result; based on the similarity result, determining the target material data corresponding to the demonstration file from the reference material data.

[0015] Furthermore, after the step of determining whether the query result contains reference material data, the method further includes:

[0016] If the query result does not contain reference material data, real-time material data corresponding to multiple target keywords are crawled from the network; the real-time material data is deduplicated to obtain initial material data; the initial material data is clustered to obtain clustering results of the initial material data; based on the clustering results, the initial material data is integrated using a data fusion algorithm to obtain target material data of the presentation file.

[0017] Furthermore, the step of fusing the demonstration file with the target material data to generate a video corresponding to the demonstration file specifically includes:

[0018] The target material data is subjected to multimodal fusion processing to obtain the material feature vector of the target material; based on the text data and the material feature vector, subtitle reference information matching the text data is generated through a sequence-to-sequence model; based on the structure data and the material feature vector, the temporal relationship and spatial layout relationship between the file content in the demonstration file and the target material is learned through a graph neural network to generate dynamic attribute information; based on the demonstration file, subtitle reference information and dynamic attribute information, a video synthesis model is used to generate the video corresponding to the demonstration file.

[0019] In order to solve the above technical problems, the embodiment of the present application further provides a video generation device, which adopts the following technical solution:

[0020] File acquisition module, used to acquire demonstration files;

[0021] A keyword extraction module is used to perform keyword extraction processing on the demonstration file to obtain multiple target keywords of the demonstration file;

[0022] A first data acquisition module, used for acquiring target material data of a presentation file based on a plurality of target keywords;

[0023] The data fusion module is used to fuse the demonstration file and the target material data to generate a video corresponding to the demonstration file.

[0024] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the above video generation method are implemented.

[0025] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable at least one processor to perform the steps of the above video generation method.

[0026] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: by extracting text data and structural data from the presentation file and combining it with rich material data, the content level and visual expression of the video can be greatly enriched. The fusion of this presentation file and the target material data is not limited to static text and pictures, but also includes dynamic background music, special effects, animation, etc., to achieve the generation of high-quality videos and make the generated videos more vivid and fascinating. Moreover, from keyword extraction of presentation files to automatic matching and fusion of materials, the entire process can be automatically processed, reducing the time cost of manual editing and video production. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0029] Figure 2 It is a flowchart of a video generation method provided by the present application;

[0030] Figure 3 It is a flowchart of a video generation method provided by the present application;

[0031] Figure 4 It is a structural schematic diagram of a video generating device provided by the present application;

[0032] Figure 5 It is a structural schematic diagram of a computer device provided by this application. DETAILED DESCRIPTION

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0034] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0036] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0037] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0038] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0039] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0040] It should be noted that the video generating method provided in the embodiment of the present application is generally executed by a server, and accordingly, the video generating device is generally arranged in the server.

[0041] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0042] Continue to refer Figure 2 , shows a flow chart of an embodiment of a video generation method according to the present application. The video generation method comprises the following steps:

[0043] Step S201, obtaining a demonstration file.

[0044] In this embodiment, the video generation method is executed on the electronic device (eg Figure 1The server shown in the figure) can obtain the presentation file through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultrawideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0045] Among them, a presentation file refers to a file created by a multimedia presentation tool. It integrates multiple content forms, such as text, pictures, charts, audio, video, etc., and presents them in the form of slides, making the presentation content more diverse. By adding animation effects, transition effects, etc., presentation files can achieve dynamic presentation of content and attract the audience's attention. In daily work and life, presentation files are widely used in teaching and training, work reports, product promotions and other occasions.

[0046] Step S202: extract keywords from the presentation file to obtain multiple target keywords of the presentation file.

[0047] Among them, keyword extraction processing is a technology that extracts the most representative words or phrases from text data parsed from a presentation file.

[0048] Among them, the target keywords are keywords that represent the core content, theme and key information of the presentation file. These target keywords are not only closely related to the content of the presentation file, but also can accurately reflect the main idea and intention of the presentation file.

[0049] In one embodiment, the presentation file can be parsed to obtain text data and structural data, and then the keywords of the text data can be identified, that is, the extracted text data can be segmented to obtain the segmentation results of the text content. According to the segmentation results, the term frequency-inverse document frequency algorithm (Term Frequency-Inverse Document Frequency, TF-IDF) is used to extract keywords from the segmented words to obtain multiple keywords. Then, the multiple keywords are screened according to the structural data to obtain multiple target keywords.

[0050] Step S203: acquiring target material data of the presentation file based on the multiple target keywords.

[0051] The target material data refers to the material data that is highly relevant to the given target keywords and meets specific needs or purposes, retrieved, screened and obtained from the material library or the network according to the given multiple target keywords. These material data can include but are not limited to pictures, videos, audios, documents, code snippets and other forms.

[0052] In one embodiment, if Figure 3 As shown, the acquired demonstration file is parsed, and keywords are identified in the text data obtained after the parsing. After obtaining multiple target keywords, the multiple target keywords can be used as query information to retrieve the target material data of the demonstration file from the local material library. When the target material library as the demonstration file cannot be retrieved in the material library, crawler technology can be used to crawl network materials from the network to obtain target material data based on the network materials. The target material data crawled from the network can be saved to the material library to update the data in the material library. Then the demonstration file and the target material data are merged to generate a video corresponding to the demonstration file.

[0053] Step S204: merge the demonstration file and the target material data to generate a video corresponding to the demonstration file.

[0054] Fusion refers to the organic combination of the content in the presentation file with the acquired target material data to create a video corresponding to the presentation file. For example, the static elements such as text, pictures, and charts in the presentation file are seamlessly connected with the dynamic target material data (such as video clips, animation effects, background music, etc.) so that they together form a coherent, vivid, and expressive video content.

[0055] In one embodiment, feature information of target material data may be extracted to form a material feature vector. Then, based on the text data and the material feature vector, subtitle reference information matching the text data of the presentation file is generated. Then, based on the structure data and the material feature vector, dynamic attribute information is generated through a graph neural network. Finally, based on the presentation file, subtitle reference information, and dynamic attribute information, a video corresponding to the presentation file may be generated using a video synthesis model.

[0056] In one example, the file content of a demonstration file is taken as an example to explain the preventive measures for a common disease, early symptom identification, the importance of vaccination, etc. The demonstration file can be parsed to obtain text data and structural data. Then, the text data is subjected to keyword extraction processing, and multiple keywords extracted by the keyword extraction processing are screened based on the structural data to obtain multiple target keywords. For example, target keywords such as "influenza prevention", "vaccination", and "personal hygiene habits" are obtained. Then, based on "influenza prevention", "vaccination", and "personal hygiene habits", the target material data can be first obtained from the local material library. If there is no corresponding material data in the local material library, the crawler technology can be used to crawl the material data corresponding to "influenza prevention", "vaccination", and "personal hygiene habits" from the network. Specifically, disease prevention posters corresponding to "influenza prevention", "vaccination", and "personal hygiene habits", demonstration pictures of correct hand washing methods, official propaganda video clips released by the Center for Disease Control and Prevention, expert interview videos, transition effects and animation effects of historical disease prevention propaganda demonstration files, etc. can be obtained from the local material library as target material data. Then, the demonstration file and the obtained target material data are merged to generate a disease prevention propaganda video. The disease prevention propaganda video is the video corresponding to the demonstration file.

[0057] In one example, the content of a presentation file includes the current economic situation, industry trends, recommended investment portfolios and expected returns. The presentation file can be parsed to obtain text data and structural data. Then, the text data is subjected to keyword extraction processing, and multiple keywords extracted by the keyword extraction processing are screened based on the structural data to obtain multiple target keywords. For example, target keywords such as "economic growth", "stock market fluctuations" and "asset allocation" are obtained. Then, based on "economic growth", "stock market fluctuations" and "asset allocation", the target material data can be first obtained from the local material library. If there is no corresponding material data in the local material library, the crawler technology can be used to crawl the material data corresponding to "economic growth", "stock market fluctuations" and "asset allocation" from the network. Specifically, historical economic data charts, company financial statement summaries, market trend charts, interpretation videos of well-known analysts, transition effects and animation effects of historical investment strategy presentation files corresponding to "economic growth", "stock market fluctuations" and "asset allocation" can be obtained from the local material library as target material data, and then the presentation file and the obtained target material data are merged to generate an investment strategy report video. The investment strategy report video is the video corresponding to the presentation file.

[0058] The embodiment of the present application can greatly enrich the content level and visual expression of the video by extracting text data and structural data from the presentation file and combining it with rich material data. The fusion of this presentation file and the target material data is not limited to static text and pictures, but also includes dynamic background music, special effects, animation, etc., to achieve the generation of high-quality videos and make the generated videos more vivid and fascinating. In addition, from keyword extraction of presentation files to automatic matching and fusion of materials, the entire process can be automatically processed, reducing the time cost of manual editing and video production.

[0059] In some optional implementations of this embodiment, in step 202, keyword extraction processing is performed on the presentation file to obtain multiple target keywords of the presentation file, which specifically includes the following steps:

[0060] The demonstration file is parsed to obtain structural data and text data of the demonstration file; the text data is subjected to keyword extraction processing to obtain multiple candidate keywords of the demonstration file; based on the structural data, the multiple candidate keywords are screened to obtain multiple target keywords of the demonstration file.

[0061] In one example, a parsing tool can be used to parse the presentation file, and by reading the layout information of each page of the presentation file, different types of elements such as text boxes, pictures, and tables can be extracted to obtain the structural data of the presentation file. At the same time, the text content in each page of the presentation file is extracted, and irrelevant characters are filtered out using regular expressions to obtain text data. When extracting keywords from text data, the TF-IDF algorithm can be used to calculate the importance score of each word, and multiple words with the highest scores can be selected as candidate keywords. Then, based on the structural data, the frequency of occurrence of candidate keywords in different locations such as text boxes, picture titles, and table titles is analyzed. Candidate keywords that appear in important locations and have a frequency exceeding a preset number of times can be selected as target keywords.

[0062] The embodiment of the present application can automatically parse the presentation file to quickly extract key information and content from the file, avoiding the tedious process of manual reading and extraction, and improving work efficiency. The keyword extraction technology can accurately identify the most representative words or phrases from a large amount of text. These keywords can reflect the core content and theme of the document. Combined with the structural data of the presentation file, such as titles, paragraphs, charts, etc., target keywords closely related to the theme and structure of the presentation file can be further screened to ensure the accuracy and pertinence of the information.

[0063] In some optional implementations of this embodiment, before acquiring the target material data of the presentation file based on the multiple target keywords in step S203, the following steps may also be included:

[0064] Acquire reference material data, perform feature extraction on the reference material data, and obtain feature information of the reference material; classify the reference material data using a clustering algorithm based on the feature information to obtain material data sets of different categories; construct index information based on the material data sets of each category; and construct a material library based on the reference material data and the index information.

[0065] In one example, when obtaining reference material data, multiple, for example, 10,000 pictures can be crawled from the Internet as a material data set. When extracting feature information of the material data, a convolutional neural network can be used to extract the feature vector of each picture. Based on the extracted feature vector, the K-Means clustering algorithm is used to classify the material pictures, and the cluster number K is set to a preset cluster value (for example, 20). After multiple iterations (for example, 100 times), the preset cluster value of different theme picture categories can be obtained, that is, the material data set is obtained. For each category of the material data set, key fields can be extracted as index fields. These fields are representative and can accurately reflect the content and characteristics of the material data set. For example, for a material data set composed of image materials, the theme, color, style, etc. of the image can be selected as index fields. For a material data set composed of text materials, keywords, subject terms, authors, etc. can be selected as index fields. Then, a suitable index construction algorithm can be selected based on the index field to generate index information, which is convenient for subsequent similar image retrieval. Finally, the classified image data and index information are stored in the database together to build a material library. For example, when clustering animal pictures, you may get different types of picture categories such as cats, dogs, and birds. By calculating the similarity of picture features in each category, the five pictures with the highest similarity are selected to build an index. When a user searches for cat pictures, they can quickly find the five most similar cat pictures in the material library. It should be understood that the above is just an example of the construction method of the material library using the reference material data as pictures. The reference material data implemented in this application may include but is not limited to pictures, videos, audio, documents, code snippets, and other forms.

[0066] The embodiment of the present application converts the original material data into feature information with clear physical or statistical significance through feature extraction, which is convenient for subsequent processing and analysis. The application of clustering algorithm enables a large amount of messy material data to be automatically classified according to its characteristics to form an orderly data set, thereby improving the management efficiency of material data. By calculating the similarity between feature vectors, index information can be generated for each category of material data set. The material library based on index information supports rapid retrieval and positioning of required materials, which improves the convenience of users to find and use materials. Before obtaining the target material of the presentation file, building a material library can provide stable and reliable material access to ensure the quality of the video corresponding to the generated presentation file.

[0067] In some optional implementations of this embodiment, the above step of “building a material library based on reference material data and index information” specifically includes the following steps:

[0068] According to the preset time interval, crawl the target network data from the network; based on the preset material crawling rules, parse and extract the crawled network data to obtain the original material data; determine whether the extracted original material data exists in the material library. If not, add the original material data to the material library.

[0069] In one example, a fixed time period every day can be set to trigger a scheduled task. Crawl the latest web page data from the website, such as blog articles, news information, etc. Parse the crawled web page data, i.e. the target network data, to extract valid materials such as text and pictures. The extracted text can be filtered out of useless information through regular expressions, and the extracted pictures can be processed by image processing, such as compression and cropping, to obtain the original material data. Then, the extracted original material data is compared with the existing reference material data in the material library to determine whether the original material already exists in the material library. If not, add it to the material library. The material library can use a database for storage, which is convenient for addition, deletion, modification and query operations. Through regular crawling and updating, the material library is continuously enriched and improved to provide more material support for subsequent video generation.

[0070] The embodiment of the present application can crawl at preset time intervals to ensure that the data in the material library is kept updated and the latest information and trends on the network are reflected in a timely manner. The automated crawling and parsing process reduces manual intervention and improves the efficiency of data collection. This not only saves labor costs, but also shortens the cycle of data updates, making data more real-time and available. Before adding the original material data to the material library, it is first determined whether it exists. This step effectively avoids repeated addition of data and optimizes the storage structure and query efficiency of the material library. At the same time, it also reduces unnecessary waste of storage space.

[0071] In some optional implementations of this embodiment, step S203, based on multiple target keywords, obtaining target material data of the presentation file, specifically includes the following steps:

[0072] Taking multiple target keywords as query information, obtaining query results corresponding to the query information from the material library; determining whether the query results contain reference material data; if the query results contain reference material data, obtaining multiple material feature information of the reference material data; extracting key feature information of multiple target keywords, performing information fusion on the key feature information, and obtaining target key feature information; calculating the similarity between the target key feature information and multiple material key feature information, and obtaining a similarity result; based on the similarity result, determining the target material data corresponding to the demonstration file from the reference material data.

[0073] In one example, multiple keywords are used as query information, and query results corresponding to the query information are obtained from the material library. It is determined whether the query result contains reference material data. If the query result contains reference material data, the material feature information of each material data can be obtained to obtain multiple material feature information. Then the keyword feature information of each target keyword is extracted, and these keyword feature information are fused by weighted averaging, feature splicing, feature mapping, etc. to obtain target key feature information that can characterize the semantics of multiple target keywords. Then, a suitable similarity calculation method (such as Euclidean distance, cosine similarity, etc.) is used to calculate the similarity between the target key information and each material feature information to obtain a similarity result. According to the obtained similarity results, the obtained similarities can be sorted in descending order, and the reference material data corresponding to the first few similarities can be used as the target material data of the presentation file.

[0074] The application embodiment can more accurately locate the material data that is highly relevant to these keywords by using multiple target keywords as query information. Compared with a single keyword query, this can reduce the interference of irrelevant results and improve the efficiency and accuracy of retrieval. In the case where there is a large amount of material data in the material library, the solution can efficiently filter out the material data that is most relevant to the target keyword, avoiding waste and redundancy of resources. At the same time, through information fusion and similarity calculation, the feature information of the material data can be more fully utilized to achieve optimal allocation of resources.

[0075] In some optional implementations of this embodiment, after the above step of “determining whether the query result contains reference material data”, the following steps may also be included:

[0076] If the query result does not contain reference material data, real-time material data corresponding to multiple target keywords are crawled from the network; the real-time material data is deduplicated to obtain initial material data; the initial material data is clustered to obtain clustering results of the initial material data; based on the clustering results, the initial material data is integrated using a data fusion algorithm to obtain target material data of the presentation file.

[0077] In one example, if the query result does not contain reference material data, multiple target keywords can be set as query conditions, and crawler technology can be used to obtain network data such as text, pictures, videos, audio, etc. related to multiple target keywords from the network, and these network data are used as real-time material data. Then, the real-time material data is deduplicated, and duplicate data is filtered out to obtain the filtered data as the initial material data. For example, the feature information of the real-time material data can be extracted, and then the feature similarity between these feature information can be calculated to obtain the similarity result. According to the similarity result, one of the two similar real-time material data can be filtered out to obtain the initial material data. When the initial material data is clustered, for example, the K-means clustering algorithm is used to cluster the initial material data, and materials with similar features or themes can be classified into one category. Then, the data fusion algorithm is used for integration, that is, the network data from different sources or different forms are integrated and fused, and the target material data of the presentation file can be obtained.

[0078] The embodiment of the present application can ensure the acquisition of the latest and most comprehensive information related to the target keyword by crawling real-time material data from the network. Deduplication processing ensures the uniqueness and purity of the material data and avoids the interference of duplicate information. Clustering processing classifies similar materials into one category to facilitate subsequent data integration and analysis. These two steps jointly optimize the quality and structure of the material data and provide a better foundation for subsequent data fusion.

[0079] In some optional implementations of this embodiment, step S204, fusing the demonstration file and the target material data to generate a video corresponding to the demonstration file, specifically includes the following steps:

[0080] The target material data is subjected to multimodal fusion processing to obtain the material feature vector of the target material; based on the text data and the material feature vector, subtitle reference information matching the text data is generated through a sequence-to-sequence model; based on the structure data and the material feature vector, the temporal relationship and spatial layout relationship between the file content in the demonstration file and the target material is learned through a graph neural network to generate dynamic attribute information; based on the demonstration file, subtitle reference information and dynamic attribute information, a video synthesis model is used to generate the video corresponding to the demonstration file.

[0081] In one example, the target material data is subjected to multimodal fusion processing. Using a pre-trained cross-modal fusion model, material data of different modalities such as text, image, and video are mapped to the same semantic space, and a material feature vector can be obtained. Then, based on the text data and the material feature vector, a sequence to sequence model (Sequence to Sequence, Seq2Seq) can be used to generate subtitle reference information that matches the text data. A pre-trained text summary model can be used to generate subtitle reference information with text data and material feature vectors as input to ensure that the semantic consistency of the subtitle information with the original text reaches above a preset threshold. Then, based on the structural data and the material feature vector, the temporal relationship and spatial layout relationship between the file content in the presentation file and the target material are learned through a graph neural network to generate dynamic attribute information. A directed graph with presentation file pages as nodes and page switching as edges is constructed, and the features of nodes and edges are encoded using a graph convolutional network (Graph Convolutional Networks, GCN) to learn the temporal dependencies of different page contents. At the same time, Convolutional Neural Networks (CNN) is used to analyze the layout of each page, extract the spatial position coordinates of different materials, and learn the spatial layout relationship of the materials. Finally, based on the demonstration file, reference subtitle information, and dynamic attribute information, a video synthesis model is used to generate the video corresponding to the demonstration file. Using a video synthesis framework based on Generative Adversarial Networks (GAN), the pages of the demonstration file are input in chronological order, and the video generation process is controlled in combination with subtitle information and dynamic attributes, so that the video screen is synchronized with the demonstration content, and a video with a duration consistent with the playback time of the demonstration file is generated. Finally, a video file that matches the content of the demonstration file and synchronizes the audio and video can be output.

[0082] The embodiment of the present application can generate a material feature vector containing multiple information sources by fusing multiple types of data such as images, text, and audio. This feature vector is more comprehensive and accurate than the feature vector of a single modality, and can better reflect the essential characteristics of the material. Through the sequence-to-sequence model, the text data can be associated with the material feature vector to generate subtitle reference information matching the text data. The graph neural network can learn the temporal relationship and spatial layout relationship between the file content and the target material in the presentation file. This relationship modeling helps to understand the overall structure and layout of the presentation file, and provides an important basis for subsequent video synthesis. Based on the learned temporal and spatial relationships, dynamic attribute information such as animation effects, transition effects, etc. can be generated. These dynamic attribute information can enhance the visual effect and attractiveness of the video. The video synthesis model comprehensively fuses the presentation file, subtitle reference information, and dynamic attribute information to generate the final presentation file video. This process fully considers the content of the presentation file, the accuracy of the subtitles, and the visual effect of the video.

[0083] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned demonstration files and the videos generated by the demonstration files, target material data, and the fusion of the demonstration files and the target material data, the above-mentioned demonstration files and the videos generated by the demonstration files, target material data, and the fusion of the demonstration files and the target material data can also be stored in a node of a blockchain.

[0084] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0085] The embodiments of the present application can acquire and process the presentation file and the target material data based on the relevant algorithms of the natural language processing technology (NLP) under artificial intelligence. Among them, artificial intelligence (AI) is the theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0086] Natural language processing technology is a key branch in the field of artificial intelligence. It focuses on giving computers the ability to understand and process human language, so that they can parse, understand and generate natural language text like humans. By simulating some core mechanisms of human language processing, NLP technology can deeply process, analyze and interpret text data, extract key information from it, understand semantic content, and perform complex tasks or generate natural language responses based on it. Natural language processing technology can be used to extract keywords from presentation files, obtain target material data, and generate videos based on presentation files and target material data.

[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0088] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0089] Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of a video generation device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0090] like Figure 4 As shown, the video generation device 400 of this embodiment includes: a file acquisition module 401, a keyword extraction module 402, a first data acquisition module 403 and a data fusion module 404. Among them:

[0091] The file acquisition module 401 is used to acquire the demonstration file;

[0092] The keyword extraction module 402 is used to perform keyword extraction processing on the presentation file to obtain multiple target keywords of the presentation file;

[0093] A first data acquisition module 403, used to acquire target material data of a presentation file based on a plurality of target keywords;

[0094] The data fusion module 404 is used to fuse the demonstration file and the target material data to generate a video corresponding to the demonstration file.

[0095] The embodiment of the present application can greatly enrich the content level and visual expression of the video by extracting text data and structural data from the presentation file and combining it with rich material data. The fusion of this presentation file and the target material data is not limited to static text and pictures, but also includes dynamic background music, special effects, animation, etc., to achieve the generation of high-quality videos and make the generated videos more vivid and fascinating. In addition, from keyword extraction of presentation files to automatic matching and fusion of materials, the entire process can be automatically processed, reducing the time cost of manual editing and video production.

[0096] In one embodiment, the keyword extraction module 402 includes:

[0097] The file parsing submodule is used to parse the demonstration file to obtain the structural data and text data of the demonstration file;

[0098] The extraction submodule is used to perform keyword extraction processing on the text data to obtain multiple candidate keywords for the presentation file;

[0099] The screening submodule is used to screen multiple candidate keywords based on the structural data to obtain multiple target keywords of the presentation file.

[0100] The embodiment of the present application can automatically parse the presentation file to quickly extract key information and content from the file, avoiding the tedious process of manual reading and extraction, and improving work efficiency. The keyword extraction technology can accurately identify the most representative words or phrases from a large amount of text. These keywords can reflect the core content and theme of the document. Combined with the structural data of the presentation file, such as titles, paragraphs, charts, etc., target keywords closely related to the theme and structure of the presentation file can be further screened to ensure the accuracy and pertinence of the information.

[0101] In one embodiment, the video generating device may further include:

[0102] The second data acquisition module is used to acquire reference material data, perform feature extraction processing on the reference material data, and obtain feature information of the reference material;

[0103] A classification module is used to classify the reference material data using a clustering algorithm according to feature information to obtain material data sets of different categories;

[0104] An information building module, used for building index information based on a material data set of each category;

[0105] The material library construction module is used to construct a material library based on reference material data and index information.

[0106] The embodiment of the present application converts the original material data into feature information with clear physical or statistical significance through feature extraction, which is convenient for subsequent processing and analysis. The application of clustering algorithm enables a large amount of messy material data to be automatically classified according to its characteristics to form an orderly data set, thereby improving the management efficiency of material data. By calculating the similarity between feature vectors, index information can be generated for each category of material data set. The material library based on index information supports rapid retrieval and positioning of required materials, which improves the convenience of users to find and use materials. Before obtaining the target material of the presentation file, building a material library can provide stable and reliable material access to ensure the quality of the video corresponding to the generated presentation file.

[0107] In one embodiment, the video generating device may further include:

[0108] A crawling module is used to crawl target network data from the network according to a preset time interval;

[0109] The third data acquisition module is used to parse and extract the crawled network data based on the preset material crawling rules to obtain the original material data;

[0110] The adding module is used to determine whether the extracted original material data exists in the material library. If not, the original material data is added to the material library.

[0111] The embodiment of the present application crawls at preset time intervals to ensure that the data in the material library is kept updated and the latest information and trends on the network are reflected in a timely manner. The automated crawling and parsing process reduces manual intervention and improves the efficiency of data collection. This not only saves labor costs, but also shortens the cycle of data updates, making data more real-time and available. Before adding the original material data to the material library, it is first determined whether it exists. This step effectively avoids repeated addition of data and optimizes the storage structure and query efficiency of the material library. At the same time, it also reduces unnecessary waste of storage space.

[0112] In one embodiment, the first data acquisition module 403 includes:

[0113] The result acquisition submodule is used to use multiple target keywords as query information and obtain query results corresponding to the query information from the material library;

[0114] A judgment submodule is used to judge whether the query result contains reference material data;

[0115] An extraction submodule, for obtaining a plurality of material feature information of the reference material data if the query result includes the reference material data;

[0116] The classification submodule is used to extract key feature information of multiple target keywords, fuse the key feature information to obtain target key feature information, calculate the similarity between the target key feature information and key feature information of multiple materials, obtain the similarity result, and determine the target material data corresponding to the demonstration file from the reference material data based on the similarity result.

[0117] The embodiment of the present application can more accurately locate the material data that is highly relevant to these keywords by using multiple target keywords as query information. Compared with a single keyword query, this can reduce the interference of irrelevant results and improve the efficiency and accuracy of retrieval. In the case of a large amount of material data in the material library, the solution can efficiently filter out the material data that is most relevant to the target keyword, avoiding waste and redundancy of resources. At the same time, through information fusion and similarity calculation, the feature information of the material data can be more fully utilized to achieve optimal allocation of resources.

[0118] In one embodiment, the judgment submodule is also used to crawl real-time material data corresponding to multiple target keywords from the network if the query result does not contain reference material data; deduplicate the real-time material data to obtain initial material data; cluster the initial material data to obtain clustering results of the initial material data; and integrate the initial material data based on the clustering results using a data fusion algorithm to obtain target material data of the presentation file.

[0119] The embodiment of the present application can ensure the acquisition of the latest and most comprehensive information related to the target keyword by crawling real-time material data from the network. Deduplication processing ensures the uniqueness and purity of the material data and avoids the interference of duplicate information. Clustering processing classifies similar materials into one category to facilitate subsequent data integration and analysis. These two steps jointly optimize the quality and structure of the material data and provide a better foundation for subsequent data fusion.

[0120] In one embodiment, the data fusion module 404 includes:

[0121] A fusion submodule is used to perform multimodal fusion processing on the target material data to obtain a material feature vector of the target material;

[0122] A first generation submodule is used to generate subtitle reference information matching the text data through a sequence-to-sequence model according to the text data and the material feature vector;

[0123] The second generation submodule is used to learn the temporal relationship and spatial layout relationship between the file content in the presentation file and the target material through the graph neural network according to the structural data and the material feature vector, and generate dynamic attribute information;

[0124] The third generation submodule is used to generate a video corresponding to the demonstration file by using a video synthesis model based on the demonstration file, subtitle reference information and dynamic attribute information.

[0125] The embodiment of the present application can generate a material feature vector containing multiple information sources by fusing multiple types of data such as images, text, and audio. This feature vector is more comprehensive and accurate than the feature vector of a single modality, and can better reflect the essential characteristics of the material. Through the sequence-to-sequence model, the text data can be associated with the material feature vector to generate subtitle reference information matching the text data. The graph neural network can learn the temporal relationship and spatial layout relationship between the file content and the target material in the presentation file. This relationship modeling helps to understand the overall structure and layout of the presentation file, and provides an important basis for subsequent video synthesis. Based on the learned temporal and spatial relationships, dynamic attribute information such as animation effects, transition effects, etc. can be generated. These dynamic attribute information can enhance the visual effect and attractiveness of the video. The video synthesis model comprehensively fuses the presentation file, subtitle reference information, and dynamic attribute information to generate the final presentation file video. This process fully considers the content of the presentation file, the accuracy of the subtitles, and the visual effect of the video.

[0126] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0127] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 having a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0128] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0129] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of the video generation method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0130] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for running a video generation method.

[0131] The network interface 63 may include a wireless network interface or a wired network interface, and the network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0132] The embodiment of the present application can greatly enrich the content level and visual expression of the video by extracting text data and structural data from the presentation file and combining it with rich material data. The fusion of this presentation file and the target material data is not limited to static text and pictures, but also includes dynamic background music, special effects, animation, etc., to achieve the generation of high-quality videos and make the generated videos more vivid and fascinating. In addition, from keyword extraction of presentation files to automatic matching and fusion of materials, the entire process can be automatically processed, reducing the time cost of manual editing and video production.

[0133] The present application also provides another implementation, namely, providing a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the video generation method as described above.

[0134] The embodiment of the present application can greatly enrich the content level and visual expression of the video by extracting text data and structural data from the presentation file and combining it with rich material data. The fusion of this presentation file and the target material data is not limited to static text and pictures, but also includes dynamic background music, special effects, animation, etc., to achieve the generation of high-quality videos and make the generated videos more vivid and fascinating. In addition, from keyword extraction of presentation files to automatic matching and fusion of materials, the entire process can be automatically processed, reducing the time cost of manual editing and video production.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0136] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A video generation method, characterized in that: The steps include: Get the demo file; Performing keyword extraction processing on the demonstration file to obtain multiple target keywords of the demonstration file; Based on the multiple target keywords, acquiring target material data of the presentation file; The demonstration file and the target material data are merged to generate a video corresponding to the demonstration file.

2. The method according to claim 1, characterized in that The step of performing keyword extraction processing on the demonstration file to obtain a plurality of target keywords of the demonstration file specifically includes: Parsing the demonstration file to obtain structural data and text data of the demonstration file; Performing keyword extraction processing on the text data to obtain a plurality of candidate keywords for the presentation file; Based on the structural data, the multiple candidate keywords are screened to obtain multiple target keywords of the presentation file.

3. The method according to claim 1, characterized in that Before the step of acquiring the target material data of the presentation file based on the multiple target keywords, the method specifically includes: Acquire reference material data, perform feature extraction processing on the reference material data, and obtain feature information of the reference material; According to the feature information, the reference material data is classified by using a clustering algorithm to obtain material data sets of different categories; Building index information based on the material data set of each category; A material library is constructed based on the reference material data and the index information.

4. The method according to claim 3, characterized in that After the step of constructing a material library based on the reference material data and the index information, the method further includes: Crawl target network data from the network according to the preset time interval; Based on the preset material crawling rules, the crawled network data is parsed and extracted to obtain the original material data; It is determined whether the extracted original material data exists in the material library. If not, the original material data is added to the material library.

5. The method according to claim 3, characterized in that: The step of acquiring the target material data of the presentation file based on the multiple target keywords specifically includes: Using the multiple target keywords as query information, obtaining query results corresponding to the query information from the material library; Determining whether the query result contains reference material data; If the query result includes the reference material data, obtaining a plurality of material feature information of the reference material data; Extracting key feature information of the multiple target keywords, and fusing the key feature information to obtain target key feature information; Calculating the similarity between the target key feature information and the plurality of material key feature information to obtain a similarity result; Based on the similarity result, target material data corresponding to the presentation file is determined from the reference material data.

6. The method according to claim 5, characterized in that After the step of determining whether the query result contains reference material data, the method further includes: If the query result does not include reference material data, crawling real-time material data corresponding to the multiple target keywords from the network; Deduplication processing is performed on the real-time material data to obtain initial material data; Performing clustering processing on the initial material data to obtain a clustering result of the initial material data; Based on the clustering result, a data fusion algorithm is used to integrate the initial material data to obtain target material data of the demonstration file.

7. The method according to claim 2, characterized in that The step of fusing the demonstration file with the target material data to generate a video corresponding to the demonstration file specifically includes: Performing multimodal fusion processing on target material data to obtain a material feature vector of the target material; Generate subtitle reference information matching the text data through a sequence-to-sequence model according to the text data and the material feature vector; According to the structural data and the material feature vector, the temporal relationship and spatial layout relationship between the file content in the demonstration file and the target material are learned through a graph neural network to generate dynamic attribute information; Based on the demonstration file, the subtitle reference information and the dynamic attribute information, a video synthesis model is used to generate a video corresponding to the demonstration file.

8. A video generating device, characterized in that: include: File acquisition module, used to acquire demonstration files; A keyword extraction module, used for performing keyword extraction processing on the demonstration file to obtain a plurality of target keywords of the demonstration file; A first data acquisition module, configured to acquire target material data of the presentation file based on the multiple target keywords; The data fusion module is used to fuse the demonstration file with the target material data to generate a video corresponding to the demonstration file.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the video generation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the video generation method according to any one of claims 1 to 7 are implemented.

Images