Text-based video generation method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence, and relates to a text-based video generation method, which comprises the following steps: acquiring a target text for generating a video, and splitting according to semantics of the target text to obtain a plurality of text paragraphs; obtaining material pictures according to semantics of the text paragraphs to obtain a material picture set; matching a video template according to the text semantics of the target text to obtain a target video template; and calculating the voice duration of each text paragraph, and performing video coding on the pictures in the material picture set according to the voice duration and the target video template to generate a target video of the target text. The invention further provides a text-based video generation device, computer equipment and a storage medium. The content matching degree of the generated video and the target text is improved, the automation degree is high, and the video production efficiency is greatly improved.

Description

Technical Field

[0001] The present application relates to the technical field of artificial intelligence and text-based video generation, and in particular to a text-based video generation method, apparatus, computer equipment and storage medium. Background Art

[0002] Traditional financial video production is mainly achieved through two methods. One is to use AI-driven content generation tools, but this method can only generate a single form of financial content, which makes it difficult to match the image content and text content of the video; the other is to use an automated video generation platform to generate videos through preset fixed templates or material libraries. Although these tools can automatically generate videos, the frame images in the generated videos are raw images generated by AI, so the image quality is poor, resulting in low quality of the generated videos, and low generation efficiency, and cannot be generated on a large scale. Summary of the invention

[0003] The purpose of the embodiments of the present application is to propose a text-based video generation method, apparatus, computer device and storage medium to solve the technical problem of poor generation efficiency of video generation methods in the prior art.

[0004] In order to solve the above technical problems, the embodiment of the present application provides a text-based video generation method, which adopts the following technical solution:

[0005] Obtaining a target text for generating a video, and splitting the target text according to the semantics of the target text to obtain multiple text paragraphs;

[0006] Acquire material pictures according to the semantics of the text paragraph to obtain a material picture set;

[0007] Matching the video template according to the text semantics of the target text to obtain a target video template;

[0008] The speech duration of each of the text paragraphs is calculated, and video encoding is performed on the pictures in the material picture set according to the speech duration and the target video template to generate a target video of the target text.

[0009] Furthermore, the target text is split according to its semantics to obtain multiple text paragraphs, including:

[0010] Identifying the text semantics of the target text, and performing word segmentation on the target text according to the text semantics to obtain at least one word segment;

[0011] Based on the text semantics, each word segment is combined according to the order and part of speech in the target text to obtain at least one text paragraph.

[0012] Furthermore, the acquiring of material images according to the semantics of the text paragraph to obtain a material image set includes:

[0013] Acquire a semantic feature vector of the text paragraph through semantic analysis;

[0014] Obtaining the image feature vector of each material picture in the preset material database, and calculating the cosine similarity between the semantic feature vector and all image feature vectors;

[0015] Matching pictures of the semantic feature vector are determined according to cosine similarity, and all matching pictures are collected to obtain the material picture set.

[0016] Furthermore, the determining the matching pictures of the semantic feature vector according to the cosine similarity and collecting all the matching pictures to obtain the material picture set includes:

[0017] Take the maximum value of all cosine similarities and determine whether the maximum value is less than a preset threshold;

[0018] If yes, a matching picture of the semantic feature vector is generated by using AI image generation technology;

[0019] If not, the material picture corresponding to the maximum value is used as the matching picture of the semantic feature vector.

[0020] Furthermore, the matching of the video template according to the text semantics of the target text to obtain the target video template includes:

[0021] Performing part-of-speech tagging and entity recognition on the target text to obtain text semantic features of the target text;

[0022] Obtaining the semantic features corresponding to each template in the preset video template library, and obtaining the template semantic features of all video templates;

[0023] The similarity between the text semantic features and the template semantic features is calculated, and the video template with the highest similarity is selected as the target video template.

[0024] Furthermore, the step of performing video encoding on the pictures in the material picture set according to the speech duration and the target video template to generate the target video of the target text includes:

[0025] sorting the material pictures in the material picture collection according to the order of the text paragraphs in the target text;

[0026] According to the speech duration of each text paragraph, the play time of each material picture in the material picture set is set respectively;

[0027] According to the target video template and the time of each material picture, video encoding is performed on each material picture in the material picture set to generate the target video.

[0028] Further, performing video encoding on each material picture in the material picture set to generate the target video includes:

[0029] Obtain the corresponding video format and playback parameters based on the terminal platform type and device performance;

[0030] According to the video format and playback parameters, an adaptive video encoding algorithm is used to dynamically adjust the video encoding parameters, and video encoding is performed on each material picture in the material picture set according to the new video encoding parameters to generate the target video.

[0031] In order to solve the above technical problems, the embodiment of the present application further provides a text-based video generation device, which adopts the following technical solution:

[0032] A splitting module, used to obtain a target text for generating a video, and split the target text according to the semantics of the target text to obtain multiple text paragraphs;

[0033] An acquisition module, used to acquire material images according to the semantics of the text paragraph to obtain a material image set;

[0034] A matching module, used for matching the video template according to the text semantics of the target text to obtain a target video template;

[0035] The video encoding module is used to calculate the speech duration of each text paragraph, and perform video encoding on the pictures in the material picture set according to the speech duration and the target video template to generate a target video of the target text.

[0036] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0037] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the text-based video generation method as described in any one of the above items are implemented.

[0038] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0039] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the text-based video generation method as described in any one of the above items are implemented.

[0040] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: by splitting the target text into multiple text paragraphs according to semantics, matching the material images based on the text semantics of the text paragraphs, it helps to analyze the semantics of the target text, thereby obtaining more matching material images, so that the image content of the video matches the content of the target text; in addition, there is no need to manually select material images, the degree of automation is high, the video production efficiency is significantly improved, and it can meet the needs of large-scale video production. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0043] Figure 2 A flowchart of an embodiment of a text-based video generation method according to the present application;

[0044] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S201;

[0045] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S202;

[0046] Figure 5 yes Figure 4 A flowchart of a specific implementation of step S403;

[0047] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S203;

[0048] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S204;

[0049] Figure 8 yes Figure 7 A flowchart of a specific implementation of step S703;

[0050] Fig. 9 is a structural schematic diagram of an embodiment of a text-based video generation device according to the present application;

[0051] Fig.10It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0053] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0056] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0057] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0058] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0059] It should be noted that the text-based video generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the text-based video generation device is generally set in the server / terminal device.

[0060] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0061] Continue to refer Figure 2 , shows a flow chart of an embodiment of a text-based video generation method according to the present application. The text-based video generation method comprises the following steps:

[0062] Step S201, obtaining a target text for generating a video, and splitting the target text according to the semantics of the target text to obtain a plurality of text paragraphs;

[0063] In this embodiment, the text-based video generation method is executed on an electronic device (eg, Figure 1 The server / terminal device shown) can receive and obtain the target text for generating the video through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. Among them, according to the semantics of the target text, the target text is split into multiple text paragraphs in order to better parse the text semantics of the target text. For example, the target text selects relevant texts on financial themes to generate financial information videos.

[0064] Step S202, obtaining material pictures according to the semantics of the text paragraph to obtain a material picture set;

[0065] In this embodiment, material images are obtained according to the semantics of the text paragraph, material images matching the content of the text paragraph are obtained, and material images of all text paragraphs are collected to obtain a material image set. The semantics of the text paragraph can be identified by natural language technology.

[0066] Step S203, matching the video template according to the text semantics of the target text to obtain a target video template;

[0067] In this embodiment, the text semantics of the target text are identified, the corresponding video template is matched according to the text semantics of the target text, and the overall structure of the video is automatically designed according to the video scene transition and perspective conversion settings preset in the video template.

[0068] Step S204, calculating the speech duration of each of the text paragraphs, and performing video encoding on the pictures in the material picture set according to the speech duration and the target video template to generate a target video of the target text.

[0069] In this embodiment, the speech duration of each text paragraph is calculated, and according to the speech duration of each text paragraph and the target video template, the pictures in the material picture set are video encoded to obtain the target video. Among them, the text-video alignment technology is used to divide the target text into several paragraphs according to semantics, and each paragraph matches a segment in the target video template. Through timeline synchronization and visual feature optimization, the text information is naturally integrated into the video segment to generate a semantically matched target video. Furthermore, for the generated target video, its objective indicators such as picture quality, sound quality, frame rate, etc. are evaluated to see whether they meet the preset quality threshold, and its content relevance and perception are evaluated by manual scoring to see whether they meet the subjective quality requirements. If the quality of the target video meets the standard, it is output as the final semantic matching video, otherwise it returns to the candidate video template screening step, and the suboptimal video template is selected to continue generating until the target video that meets the quality requirements is found.

[0070] This application splits the target text into multiple text paragraphs according to semantics, and matches the material images based on the text semantics of the text paragraphs. Intelligent segmentation helps to analyze the semantics of the target text, thereby obtaining more matching material images, so that the image content of the video matches the content of the target text. In addition, there is no need to manually select material images, and the degree of automation is high, which significantly improves the efficiency of video production and can meet the needs of large-scale video production.

[0071] like Figure 3 As shown, in some optional implementations of this embodiment, step S201 includes the following steps:

[0072] Step S301, identifying the text semantics of the target text, and performing word segmentation on the target text according to the text semantics to obtain at least one word segmentation;

[0073] Step S302: Based on the text semantics, the word segments are combined according to the order and part of speech of each word segment in the target text to obtain at least one text paragraph.

[0074] In this embodiment, in order to more accurately understand the text semantics of the target text, the text semantics of the target text is identified by natural language processing, and the target text is segmented according to the text semantics to obtain at least one segmentation; according to the text semantics, the segmentations are combined according to the order and part of speech of each segmentation in the target text to obtain at least one text paragraph. Among them, the word vector sequence in the text is encoded by a deep learning model such as BiLSTM to capture the contextual semantic information of the text, and a pre-trained word vector model such as Word2Vec or Glove is used to segment the target text according to the contextual semantic information. By intelligently segmenting the target text and splitting the target text into text paragraphs, it is helpful to understand the text semantics to match pictures that are more in line with the text semantics.

[0075] like Figure 4 As shown, in some optional implementations of this embodiment, step S202 includes the following steps:

[0076] Step S401, obtaining a semantic feature vector of the text paragraph through semantic analysis;

[0077] Step S402, obtaining the image feature vector of each material picture in the preset material database, and calculating the cosine similarity between the semantic feature vector and all image feature vectors;

[0078] Step S403, determining matching pictures of the semantic feature vector according to cosine similarity, and gathering all matching pictures to obtain the material picture set.

[0079] In this embodiment, in order to filter out material pictures that match the text paragraph, semantic analysis is performed on each text paragraph to obtain the semantic feature vector of the text paragraph; the image feature vectors of all material pictures in the preset material database are calculated, the cosine similarity between the semantic feature vector and all image feature vectors is calculated, the matching pictures of the semantic feature vector are determined according to the cosine similarity, and the matching pictures of all text paragraphs are collected to obtain a material picture set. By calculating the cosine similarity of the feature vector, material pictures that match the semantic feature vector of the text paragraph are effectively filtered out, so as to prepare for generating video content that better matches the target text.

[0080] like Figure 5As shown, in some optional implementations of this embodiment, step S403 includes the following steps:

[0081] Step S501, taking the maximum value of all cosine similarities, and determining whether the maximum value is less than a preset threshold;

[0082] Step S5021: If yes, generate a matching picture of the semantic feature vector by using AI image generation technology;

[0083] Step S5022: If not, the material picture corresponding to the maximum value is used as the matching picture of the semantic feature vector.

[0084] In this embodiment, in order to ensure that the material image matches the text paragraph, the similarity between the semantic feature vector of the text paragraph and the image feature vector of the material image must be greater than a certain threshold. Therefore, based on the cosine similarity between the semantic feature vector and all image feature vectors, the image feature vector with the largest cosine similarity is extracted, and it is determined whether the cosine similarity is less than the preset threshold. If the cosine similarity is less than the preset threshold, it indicates that the material image corresponding to the cosine similarity is not sufficiently matched with the text paragraph. At this time, a matching image with a higher matching degree with the semantic feature vector can be generated through AI image generation technology to obtain a matching image of the corresponding text paragraph; if the cosine similarity is not less than the preset threshold, it indicates that the material image corresponding to the cosine similarity meets the requirements for matching the text paragraph, and the material image corresponding to the cosine similarity is used as the matching image of the text paragraph corresponding to the semantic feature.

[0085] like Figure 6 As shown, in some optional implementations of this embodiment, step S203 includes the following steps:

[0086] Step S601, performing part-of-speech tagging and entity recognition on the target text to obtain text semantic features of the target text;

[0087] Step S602, obtaining semantic features corresponding to each template in the preset video template library, and obtaining template semantic features of all video templates;

[0088] Step S603, calculating the similarity between the text semantic features and the template semantic features, and selecting the video template with the highest similarity as the target video template.

[0089] In this embodiment, in video production, users can also use video templates to quickly set up picture sets, thereby further speeding up the process of video production and improving video production efficiency. Among them, video templates are pre-designed with video playback settings such as picture switching and playback perspective conversion. For each video template in the video template library, the visual features of its key frames are extracted by convolutional neural networks such as ResNet, and then the video description generation model (such as Seq2Seq) is used to generate the semantic description text of the video, extract the semantic feature vector, and construct a video semantic model. By performing part-of-speech tagging and entity recognition on the target text, the semantic information of the target text at different granularities such as paragraphs and sentences is analyzed to obtain the text semantic features of the target text; by calculating the similarity between the input text semantic features and the template semantic features of each video in the video module library, the video template with the highest similarity is selected as the target video template. In addition, the key frames, picture dwell time, special effects and other elements of the video template can be adjusted, and corresponding subtitles can be added to make the settings of the target video template more consistent with the target text.

[0090] like Figure 7 As shown, in some optional implementations of this embodiment, step S204 includes the following steps:

[0091] Step S701, sorting the material pictures in the material picture set according to the order of the text paragraphs in the target text;

[0092] Step S702, setting the play time of each material picture in the material picture set according to the voice duration of each text paragraph;

[0093] Step S703, performing video encoding on each material picture in the material picture set according to the target video template and the time of each material picture to generate the target video.

[0094] In this embodiment, in order to make the generated video more consistent with the target text, not only the material pictures of the material picture set need to be consistent with the content of the target text, but the playback order and playback time of the material pictures should also correspond to the target text. According to the order of the text paragraphs in the target text, the material pictures corresponding to the text paragraphs are sorted to determine the playback order of each material picture; through natural language processing, the voice duration of each text paragraph is calculated, and the playback time of the corresponding material picture is set according to the voice duration of each text paragraph; according to the playback order and playback time of each material picture, the material picture set is video encoded according to the target video template to obtain the target video.

[0095] like Figure 8 As shown, in some optional implementations of this embodiment, step S703 includes the following steps:

[0096] Step S801, obtaining corresponding video format and playback parameters according to the terminal platform type and device performance;

[0097] Step S802, according to the video format and the playback parameters, an adaptive video encoding algorithm is used to dynamically adjust the video encoding parameters, and video encoding is performed on each material picture in the material picture set according to the new video encoding parameters to generate the target video.

[0098] In this embodiment, according to the video format requirements and device performance of different terminal platforms, adaptive video encoding and bit rate control technology is used to perform real-time format conversion and dynamic adaptation of the generated target video, and the video format is matched with the playback platform by adjusting the encoding parameters, resolution, bit rate, frame rate, etc., and optimized according to the device display capability and network conditions to ensure smooth video playback. Among them, the encoded video data is encapsulated and format converted to generate a video file format that matches the terminal platform, such as MP4, HLS, etc.

[0099] This embodiment dynamically adjusts video encoding parameters through an adaptive video encoding algorithm according to the platform type and device performance of the terminal, thereby reducing the amount of video data while ensuring video quality, thereby achieving smooth playback of the target video.

[0100] Furthermore, in order to further improve the matching degree between the target video and the target text, the background music (i.e., BGM) can be matched according to the content of the target text. Specifically, it includes: performing sentiment analysis on the target text, judging the sentiment tendency and sentiment intensity of the target text, and obtaining the sentiment characteristics of the target text; classifying the text according to the sentiment characteristics using a sentiment classification algorithm, and obtaining the sentiment category to which the text belongs; screening out candidate background music that matches the sentiment category from a pre-built music feature library; extracting the audio features of the candidate music, including rhythm, melody, timbre, etc., through an audio analysis algorithm; calculating the similarity between the sentiment characteristics of the target text and the audio features of the candidate music, and obtaining the background music with the highest matching degree; dynamically adjusting the playback parameters of the matched background music, such as volume, speed, etc., according to the sentiment intensity of the target text; and finally outputting background music that is coordinated with the sentiment characteristics of the target text, and creating a music effect that echoes the target text.

[0101] The embodiment of the present application helps to analyze the semantics of the target text by splitting the target text into multiple text paragraphs according to semantics, and matching the material images based on the text semantics of the text paragraphs, so as to obtain more matching material images, so that the image content of the video matches the content of the target text; in addition, there is no need to manually select material images, and the degree of automation is high, which significantly improves the efficiency of video production and can meet the needs of large-scale video production.

[0102] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0103] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0104] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0105] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0106] Further references Fig. 9 , as a response to the above Figure 2 The present application provides an embodiment of a text-based video generation device, and the device embodiment is Fig. 9 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0107] like Fig. 9As shown, the text-based video generation device 900 described in this embodiment includes: a splitting module 901, an acquisition module 902, a matching module 903, and a video encoding module 904.

[0108] in:

[0109] The splitting module 901 is used to obtain a target text for generating a video, and split the target text according to the semantics of the target text to obtain multiple text paragraphs;

[0110] An acquisition module 902 is used to acquire material pictures according to the semantics of the text paragraph to obtain a material picture set;

[0111] A matching module 903 is used to match the video template according to the text semantics of the target text to obtain a target video template;

[0112] The video encoding module 904 is used to calculate the speech duration of each of the text paragraphs, and perform video encoding on the pictures in the material picture set according to the speech duration and the target video template to generate a target video of the target text.

[0113] In some optional implementations of this embodiment, the splitting module 901 includes:

[0114] A word segmentation unit, used for identifying the text semantics of the target text, and performing word segmentation on the target text according to the text semantics to obtain at least one word segmentation;

[0115] The combining unit is used to combine the words according to the order and part of speech of each word in the target text based on the text semantics to obtain at least one text paragraph.

[0116] In some optional implementations of this embodiment, the acquisition module 902 includes:

[0117] An acquisition unit, used for acquiring a semantic feature vector of the text paragraph through semantic analysis;

[0118] The first similarity calculation is used to obtain the image feature vector of each material picture in the preset material database, and calculate the cosine similarity between the semantic feature vector and all image feature vectors;

[0119] The picture matching unit is used to determine the matching pictures of the semantic feature vector according to the cosine similarity, and collect all the matching pictures to obtain the material picture set.

[0120] In some optional implementations of this embodiment, the picture matching unit includes:

[0121] A judging unit, used to take the maximum value among all cosine similarities and judge whether the maximum value is less than a preset threshold;

[0122] A picture generating unit, configured to generate a matching picture of the semantic feature vector by using an AI image generating technology if yes;

[0123] The determination unit is configured to, if not, use the material picture corresponding to the maximum value as the matching picture of the semantic feature vector.

[0124] In some optional implementations of this embodiment, the matching module 903 includes:

[0125] A tagging unit, used to perform part-of-speech tagging and entity recognition on the target text to obtain text semantic features of the target text;

[0126] The template feature acquisition unit is used to acquire the semantic features corresponding to each template in the preset video template library, and obtain the template semantic features of all video templates;

[0127] The second similarity calculation unit is used to calculate the similarity between the text semantic feature and the template semantic feature, and select the video template with the highest similarity as the target video template.

[0128] In some optional implementations of this embodiment, the video encoding module 904 includes:

[0129] A sorting unit, used for sorting the material pictures in the material picture set according to the order of each text paragraph in the target text;

[0130] A duration setting unit, used to set the play time of each material picture in the material picture set according to the voice duration of each text paragraph;

[0131] The video encoding unit is used to perform video encoding on each material picture in the material picture set according to the target video template and the time of each material picture to generate the target video.

[0132] In some optional implementations of this embodiment, the video encoding unit includes:

[0133] The parameter acquisition subunit is used to obtain the corresponding video format and playback parameters according to the terminal platform type and device performance;

[0134] According to the video format and playback parameters, an adaptive video encoding algorithm is used to dynamically adjust the video encoding parameters, and video encoding is performed on each material picture in the material picture set according to the new video encoding parameters to generate the target video.

[0135] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0136] The computer device 10 includes a memory 1001, a processor 1002, and a network interface 1003 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 10 having a memory 1001, a processor 1002, and a network interface 1003, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0137] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0138] The memory 1001 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 1001 can be an internal storage unit of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 1001 can also be an external storage device of the computer device 10, such as a plug-in hard disk equipped on the computer device 10, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 1001 can also include both the internal storage unit of the computer device 10 and its external storage device. In this embodiment, the memory 1001 is generally used to store the operating system and various application software installed on the computer device 10, such as computer-readable instructions of the text-based video generation method, etc. In addition, the memory 1001 can also be used to temporarily store various data that have been output or are to be output.

[0139] The processor 1002 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 1002 is generally used to control the overall operation of the computer device 10. In this embodiment, the processor 1002 is used to run the computer-readable instructions stored in the memory 1001 or process data, such as computer-readable instructions for running the text-based video generation method.

[0140] The network interface 1003 may include a wireless network interface or a wired network interface. The network interface 1003 is generally used to establish a communication connection between the computer device 10 and other electronic devices.

[0141] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the text-based video generation method as described above.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0143] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0144] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A text-based video generation method, characterized in that: include: Obtaining a target text for generating a video, and splitting the target text according to the semantics of the target text to obtain multiple text paragraphs; Acquire material pictures according to the semantics of the text paragraph to obtain a material picture set; Matching the video template according to the text semantics of the target text to obtain a target video template; The speech duration of each of the text paragraphs is calculated, and video encoding is performed on the pictures in the material picture set according to the speech duration and the target video template to generate a target video of the target text.

2. The text-based video generation method according to claim 1, characterized in that The target text is split according to its semantics to obtain multiple text paragraphs, including: Identifying the text semantics of the target text, and performing word segmentation on the target text according to the text semantics to obtain at least one word segment; Based on the text semantics, each word segment is combined according to the order and part of speech in the target text to obtain at least one text paragraph.

3. The text-based video generation method according to claim 1, characterized in that The step of acquiring the material pictures according to the semantics of the text paragraph to obtain the material picture set includes: Acquire a semantic feature vector of the text paragraph through semantic analysis; Obtaining the image feature vector of each material picture in the preset material database, and calculating the cosine similarity between the semantic feature vector and all image feature vectors; Matching pictures of the semantic feature vector are determined according to cosine similarity, and all matching pictures are collected to obtain the material picture set.

4. The text-based video generation method according to claim 3, characterized in that: The determining the matching pictures of the semantic feature vector according to the cosine similarity, and collecting all the matching pictures to obtain the material picture set, comprises: Take the maximum value of all cosine similarities and determine whether the maximum value is less than a preset threshold; If yes, a matching picture of the semantic feature vector is generated by using AI image generation technology; If not, the material picture corresponding to the maximum value is used as the matching picture of the semantic feature vector.

5. The text-based video generation method according to claim 1, characterized in that: The step of matching the video template according to the text semantics of the target text to obtain the target video template includes: Performing part-of-speech tagging and entity recognition on the target text to obtain text semantic features of the target text; Obtaining the semantic features corresponding to each template in the preset video template library, and obtaining the template semantic features of all video templates; The similarity between the text semantic features and the template semantic features is calculated, and the video template with the highest similarity is selected as the target video template.

6. The text-based video generation method according to claim 1, characterized in that: The step of performing video encoding on the pictures in the material picture set according to the speech duration and the target video template to generate the target video of the target text includes: sorting the material pictures in the material picture collection according to the order of the text paragraphs in the target text; According to the speech duration of each text paragraph, the play time of each material picture in the material picture set is set respectively; According to the target video template and the time of each material picture, video encoding is performed on each material picture in the material picture set to generate the target video.

7. The text-based video generation method according to claim 6, characterized in that: Performing video encoding on each material picture in the material picture set to generate the target video includes: Obtain the corresponding video format and playback parameters based on the terminal platform type and device performance; According to the video format and playback parameters, an adaptive video encoding algorithm is used to dynamically adjust the video encoding parameters, and video encoding is performed on each material picture in the material picture set according to the new video encoding parameters to generate the target video.

8. A text-based video generation device, characterized in that: include: A splitting module, used to obtain a target text for generating a video, and split the target text according to the semantics of the target text to obtain multiple text paragraphs; An acquisition module, used to acquire material images according to the semantics of the text paragraph to obtain a material image set; A matching module, used for matching the video template according to the text semantics of the target text to obtain a target video template; The video encoding module is used to calculate the speech duration of each text paragraph, and perform video encoding on the pictures in the material picture set according to the speech duration and the target video template to generate a target video of the target text.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the text-based video generation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the text-based video generation method according to any one of claims 1 to 7 are implemented.

Images