Video generation method and device based on video script, equipment and medium

Abstract

The invention relates to the field of video generation, and discloses a video generation method and device based on a video script, equipment and a medium, and the method comprises the steps: responding to a scene selection instruction, and confirming an application scene generated by a current video; acquiring a corresponding video script according to the application scene and a preset acquisition strategy; performing text analysis processing on the video script, and extracting text content and auxiliary information of the video script; and obtaining a preset sound sample, and generating a target video matched with the preset sound sample and the video script according to the preset sound sample and the text content and the auxiliary information of the video script. By acquiring the video scripts in different application scenes and automatically generating the video matched with the preset sound according to the video scripts, personalized and high-efficiency content production is realized, meanwhile, the consistency of themes is ensured, the video generation efficiency is improved, and the video generation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of video generation technology and financial technology, and in particular to a video generation method, device, equipment and medium based on a video script. Background Art

[0002] With the rise of short videos, videos have gradually become one of the most popular and effective forms of advertising content. In terms of marketing, videos can attract users' attention, convey information, and achieve the purpose of publicity. For example, in the financial insurance industry, more and more insurance agents use videos to introduce insurance products, interpret insurance terms, and popularize insurance types.

[0003] However, traditional video generation requires a lot of manual production processes such as shooting and editing. Insurance agents often need a lot of manpower, material resources and time to create high-quality videos during the video production and sound recording process. This is not only time-consuming and costly, but also for the financial field that needs to respond quickly to market changes and has a large number of products, the efficiency of this method is obviously unable to meet the needs. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a video generation method, device, equipment and medium based on video scripts that can be applied to financial technology or other related fields. Its main purpose is to improve the efficiency of video generation and reduce the cost of video generation.

[0005] The technical solution of the present invention is as follows:

[0006] A first aspect of the present invention provides a video generation method based on a video script, comprising:

[0007] In response to the scene selection instruction, confirming the application scene of the current video generation;

[0008] Acquire the corresponding video script according to the application scenario and the preset acquisition strategy;

[0009] Performing text analysis on the video script to extract text content and auxiliary information of the video script;

[0010] A preset sound sample is obtained, and a target video matching the preset sound sample and the video script is generated according to the preset sound sample and the text content and auxiliary information of the video script.

[0011] A second aspect of the present invention provides a video generation device based on a video script, comprising:

[0012] A scene confirmation module, used to confirm the application scene generated by the current video in response to the scene selection instruction;

[0013] A script acquisition module, used to acquire the corresponding video script according to the application scenario and a preset acquisition strategy;

[0014] A text analysis module, used to perform text analysis on the video script and extract text content and auxiliary information of the video script;

[0015] The video generation module is used to obtain a preset sound sample, and generate a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script.

[0016] A third aspect of the present invention provides a computer device, comprising at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned video script-based video generation method.

[0019] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned video script-based video generation method.

[0020] Beneficial effects: The present invention discloses a video generation method, device, equipment and medium based on a video script. Compared with the prior art, the embodiments of the present invention confirm the application scenario of the current video generation by responding to the scene selection instruction; obtain the corresponding video script according to the preset acquisition strategy according to the application scenario; perform text analysis processing on the video script to extract the text content and auxiliary information of the video script; obtain a preset sound sample, and generate a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script. Through AIGC technology, video scripts can be automatically acquired and generated according to preset scene instructions, which greatly improves the efficiency of content creation, and video content matching the preset sound sample can be generated by using the text content and auxiliary information of the video script, realizing highly personalized and customized video generation, meeting the unique needs of different users, not only reducing manual participation and reducing generation costs, but also generating a large amount of content in a short time while maintaining the consistency of the theme, meeting the needs of rapid production, and improving video generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the scheme in the present invention, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of an application environment of a video generation method based on a video script provided by an embodiment of the present invention;

[0023] Figure 2 A flow chart of a video generation method based on a video script provided by an embodiment of the present invention;

[0024] Figure 3 A flow chart of step S201 in the video generation method based on the video script provided in an embodiment of the present invention;

[0025] Figure 4 A flow chart of step S202 in the video generation method based on video script provided in an embodiment of the present invention;

[0026] Figure 5 A flow chart of step S204 in the video generation method based on the video script provided in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of functional modules of a video generation device based on a video script provided by an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0030] The video generation method based on video script provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment of FIG. 1 , it includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0031] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0032] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0033] The server 105 may be a server that provides various services, such as a background server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The background server may analyze and process the received data such as the user request, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0034] It should be noted that the video generation method based on the video script provided in the embodiment of the present application can generally be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the video generation device based on the video script provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103. Alternatively, the video generation method based on the video script provided in the embodiment of the present invention can also generally be executed by the server 105. Accordingly, the video generation device based on the video script provided in the embodiment of the present invention can generally be set in the server 105.

[0035] It should be understood that the numbers of the above terminal devices, networks and servers are only illustrative. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0036] like Figure 2 As shown, the video generation method based on the video script provided by the embodiment of the present invention specifically includes the following steps:

[0037] S201: In response to a scene selection instruction, confirm an application scene for current video generation.

[0038] In this embodiment, video generation can be applied in different scenarios of insurance companies. Users can input corresponding scenario selection instructions according to different application requirements, and the video generation system parses the instructions input by the user to determine the required video generation application scenario. For example, the application scenario can be to create a new insurance product introduction video for marketing and customer education, or it can be to reuse and clone existing videos on the platform. For example, different agents can clone existing insurance terms explanation videos to generate explanation videos with the same theme but different pictures, thereby realizing personalized presentation of content. By accurately identifying user needs, appropriate strategies and tools can be selected for the subsequent video generation process to improve the efficiency and relevance of video generation.

[0039] S202: Acquire a corresponding video script according to a preset acquisition strategy based on the application scenario.

[0040] In this embodiment, based on the application scenario selected by the current user, different acquisition strategies are used to obtain video scripts. In the field of AIGC (artificial intelligence generated content), video scripts refer to detailed plans that provide guidance and structure for the video generation process, outlining the content, structure, and style of the video. The acquired video scripts typically include scene settings, characters and dialogues, action descriptions, visual details, etc., so that the system can automatically generate video content based on these guidelines.

[0041] For different application scenarios, specific video scripts can be obtained through different acquisition methods. For example, for the scenario of creating a new insurance product introduction video, the knowledge graph and templates in the financial insurance field can be used to automatically generate an introduction script for the new insurance product. The script may include the product's coverage, price, advantages, and case studies. For the scenario of reusing existing videos on the cloning platform, video analysis can be used to extract information from them and then generate a new video script. Automated script generation and extraction can significantly reduce the workload of video production in the early stages and speed up the video generation process.

[0042] S203: Perform text analysis on the video script to extract text content and auxiliary information of the video script.

[0043] In this embodiment, for the video scripts obtained in different application scenarios, natural language processing technologies such as topic modeling, sentiment analysis and entity recognition are used to perform text analysis on the video scripts to extract the text content and auxiliary information in the video scripts. Specifically, the video scripts can be preprocessed first, including text cleaning, word segmentation, and stop word removal, to lay the foundation for subsequent processing, and then named entity recognition is performed on the video scripts to identify entities in the text, such as names, places, organizations, etc. Based on the named entity recognition results, the character dialogue content in the video script, that is, the text content that needs dubbing, is extracted.

[0044] At the same time, sentiment analysis is also performed on the video script to determine the emotional tendency in the text, such as positive, negative or neutral; and the tone in the video script is also analyzed, such as questions, statements, commands, etc., so as to obtain auxiliary information such as emotional information and intonation information in the video script. By combining auxiliary information such as emotion and intonation on the basis of obtaining the main text content, the script content can be better understood when automatically generating videos, providing guidance for the design of the visual and auditory elements of the video, and then generating more accurate video content, ensuring the quality and accuracy of the video content.

[0045] S204: Obtain a preset sound sample, and generate a target video that matches the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script.

[0046] In this embodiment, when generating a video, the user can select different sound samples according to needs, such as selecting his own voice, the voice of a designated person, a virtual synthetic voice, etc. as a sound sample, and combine the text content and auxiliary information in the video script, and generate a target video that matches the preset sound sample and the video script through AIGC video generation technology such as deep learning and generative adversarial networks (GANs). That is, the generated target video not only meets the requirements of the video script, but also makes the generated video content consistent with the script in terms of emotion and theme. This consistency improves the quality of the video content and makes the video more attractive and persuasive. At the same time, because video generation has a certain creative expansion space, the generated video has diversity, and the generated video matches the preset sound sample in sound and tone. Therefore, even based on the same video script, a generation result with different pictures and different sounds can be obtained, and a large number of differentiated videos can be generated in a short time, meeting the needs of different users to quickly generate personalized videos, enriching the diversity of video generation.

[0047] In the above embodiment, the present invention discloses a video generation method based on a video script, which confirms the application scenario of the current video generation by responding to a scene selection instruction; obtains the corresponding video script according to the application scenario according to a preset acquisition strategy; performs text analysis processing on the video script to extract the text content and auxiliary information of the video script; obtains a preset sound sample, and generates a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script. Through AIGC technology, video scripts can be automatically acquired and generated according to preset scene instructions, which greatly improves the efficiency of content creation, and video content matching the preset sound sample can be generated by using the text content and auxiliary information of the video script, realizing highly personalized and customized video generation, meeting the unique needs of different users, not only reducing manual participation and reducing generation costs, but also generating a large amount of content in a short time while maintaining the consistency of the theme, meeting the needs of rapid production, and improving video generation efficiency.

[0048] In one embodiment, Figure 3 As shown, step S201 includes:

[0049] S301, displaying a video generation interface, wherein the video generation interface includes several application scenario options;

[0050] S302: receiving a scene selection operation of the user for the plurality of application scenes, and confirming an application scene for current video generation, wherein the application scene is a new video generation scene or a general video multiplexing scene.

[0051] In this embodiment, an intuitive and user-friendly interface is created in advance to list all available video generation scenarios. The video generation interface is displayed when the video is generated. A variety of application scenario options are provided in the video generation interface. The user can enter the corresponding scenario selection operation by checking or clicking the corresponding scenario option, and then select the currently required application scenario, wherein the application scenario includes a new video generation scenario or a general video reuse scenario. For example, the video generation interface includes two categories of scenarios: a new video generation scenario and a general video reuse scenario. In the new video generation scenario, it can further include segmented scenario options such as "insurance product introduction", "risk management education", and "investment strategy explanation". In the general video reuse scenario, it can further include segmented scenario options such as "introduction to existing insurance terms", "cross-platform content reuse", and "video content update". The user interacts with the system through the interface to select the currently required application scenario. Preferably, each scenario option is associated with a preset video script template and visual element. After the current application scenario is selected and determined, the corresponding script template and visual element can be automatically called and loaded for subsequent video generation to further improve the efficiency of video generation.

[0052] In one embodiment, when the application scenario is a new video generation scenario, acquiring the corresponding video script according to the application scenario according to a preset acquisition strategy specifically includes:

[0053] Receive the script file uploaded by the user, perform text extraction processing on the script file, and obtain the corresponding video script.

[0054] In the present embodiment, when the application scenario is a new video generation scenario, the user can upload a script file (such as TXT, PDF, Word document, etc.) for generating a video through an interface or API provided by the system. After receiving the script file, the content in the script file is recognized and converted into an editable text format through optical character recognition (OCR) technology or text extraction algorithm, thereby obtaining the corresponding video script. Further, the video script is displayed to the user in a visual form, and the user can review and modify the script, and then start the subsequent video generation process according to the content of the video script after confirming that it is correct. Through automated text extraction, the time and potential errors of manual input are reduced, work efficiency is improved, and the accuracy of text extraction ensures that the content of the video script is consistent with the file uploaded by the user, ensuring the accuracy and reliability of the video content.

[0055] For example, in the field of finance and insurance, users need to make an introductory video about a new insurance product. Users can upload a script file containing information such as product features, target customers, and market advantages. The system performs text extraction on the uploaded script file, extracts and generates an outline of the video script according to the chapter title structure in the script file, including the various parts of the video and the detailed content of each part, etc. The accuracy and relevance of the video script ensure that the generated video can effectively convey the information about the financial insurance product, thereby improving the quality of the final video.

[0056] In one embodiment, Figure 4 As shown, when the application scenario is a general video multiplexing scenario, the step S202 includes:

[0057] S401, displaying a video selection interface, wherein the video selection interface includes a plurality of historically generated general videos;

[0058] S402, receiving a video selection operation of a user for a plurality of general videos, and confirming the video to be multiplexed;

[0059] S403: extract video content from the video to be multiplexed, and generate a text description of the video to be multiplexed as a video script for the current application scenario.

[0060] In this embodiment, a video selection interface is created in advance to display a general video library generated historically. For example, in the field of finance and insurance, this interface can display a series of historical videos, such as general videos on topics such as "investment risk education", "insurance claims process", and "financial product comparison". Users can select the videos they need to reuse by browsing thumbnails or video titles. This provides intuitive video previews for general videos, allowing users to quickly find what they need, so that they can directly select historical videos from the existing video library for reuse instead of creating them from scratch, thereby saving time and resources.

[0061] When a user selects a video about "investment risk education" on the video selection interface, the system will record the user's selection and provide further customization options, such as adding specific annotations or updating some data, so that users can select specific video content for reuse according to their needs, and realize flexible video one-click cloning function. When the user selects a video, the video content is extracted from the video to be reused selected by the user, and the video description (VideoCaption) technology is used to convert the visual content of the video into a natural language description. For example, through deep learning models such as convolutional neural networks (CNN) and recurrent neural networks (RNN), key information in the video is extracted and descriptive text is generated. Through automated video content extraction and generation of corresponding text descriptions as video scripts, a basis for video reuse is provided, allowing users to use existing video content on the platform for secondary generation without re-recording, thus realizing diversified presentation of content.

[0062] In one embodiment, Figure 5 As shown, step S204 includes:

[0063] S501, obtaining a corresponding audio clip from a database as a preset sound sample according to the user's identity identifier;

[0064] S502, inputting the preset sound sample and the text content and auxiliary information of the video script into a trained text-to-speech generation model, extracting the timbre features of the preset sound sample, and generating a target audio with corresponding timbre and pitch based on the text content and auxiliary information;

[0065] S503: Input the target audio and the text content of the video script into a trained video generation model, generate a corresponding video screen according to the text content, and then synthesize it with the target audio to generate a corresponding target video.

[0066] In this embodiment, different users can use personalized sound samples to make videos according to their own needs. The system identifies the user's identity based on the current user's identity identifier, such as the login username, user unique ID, etc., and retrieves and obtains the audio clip previously recorded or specified by the user from the database as the preset sound sample. For example, the voice sample of insurance agent Zhang San is stored in the system. When he needs to make a short video about family insurance planning, his voice sample can be automatically extracted as the preset sound sample, so that each user can use his own voice to generate a video, making the video more personalized.

[0067] Based on the acquired preset sound samples, as well as the text content and auxiliary information of the video script obtained by the previous analysis, the model is able to generate target audio with corresponding timbre and pitch corresponding to the text content according to the timbre characteristics of the text content and preset sound samples, combined with the emotion, intonation and other factors in the auxiliary information. For example, if the video script is about the introduction of "family insurance planning", the text-to-speech generation model will generate an audio that matches the characteristics of Zhang San's voice, and the content is about the explanation of family insurance planning, thereby generating natural, fluent and matching speech with the original script, ensuring the accuracy of the video. At the same time, the emotion and intonation of the synthesized speech can be controlled through auxiliary information to improve the personalization and expressiveness of the video content.

[0068] After generating the target audio that matches the preset sound sample, the target audio and the text content of the video script are input into a trained video generation model. The model can generate the corresponding video images based on the text content and synthesize them with the target audio to generate the final target video. For example, if the video script is about "family insurance planning", the video generation model will create a series of visual content, such as charts, animations, and related scenes, and synthesize them with the audio of Zhang San explaining family insurance planning to form a complete educational video. By combining text-to-speech and video generation technology, high-quality video content can be generated quickly, improving the efficiency of content production, and the synchronous generation of video images and audio ensures the consistency and coordination of information transmission, improving the overall quality of the video.

[0069] For example, when an insurance company needs to produce a series of promotional videos, each of which requires the dubbing of an agent, it can go through the above video generation steps, upload the corresponding video script and perform text analysis, use NLP technology to analyze the script content and auxiliary information such as emotion and intonation, and combine the agent's voice sample to generate audio that matches the agent's voice. In this way, the insurance company can generate audio for multiple videos in a short period of time, greatly improving video production efficiency and saving costs.

[0070] In one embodiment, after step S204, the method further includes:

[0071] Displaying a video preview interface, wherein the video corpus interface includes the currently generated target video;

[0072] When receiving a preview operation input by the user, playing the currently generated target video, and receiving feedback adjustment information input by the user;

[0073] The speech rate and / or intonation of the target video is adjusted according to the feedback adjustment information to generate an adjusted target video.

[0074] In this embodiment, after the video generation is completed, the system will display a video preview interface, which contains the currently generated target video. This interface can be a web page or a module within an application, which displays a thumbnail and a play button of the video. For example, if the video is an introduction to "family insurance planning", the preview interface will display the cover image and play options of the video, so that the user can immediately see the result of the video generation and provide instant feedback. When the user clicks the play button on the preview interface, the system will play the generated target video. During or after the playback, the system will provide a feedback interface, and the user can enter feedback adjustment information for the currently generated target video, such as too fast speaking speed, too bland tone, etc., so as to directly participate in the fine-tuning process of the video and make the final video more in line with expectations.

[0075] After receiving user feedback, if the user feels that the video is too fast, the system will reduce the speed to match the user's preference. If the user thinks the tone is too flat, the system can adjust the intonation of the voice to increase the expressiveness. This allows personalized adjustments based on user feedback, and flexible creation and adjustment of personalized video content to meet the needs of different customers.

[0076] In one embodiment, after step S204, the method further includes:

[0077] Performing similarity analysis on the preset sound sample and the target video to obtain the timbre similarity between the target video and the preset sound sample;

[0078] The target video is adjusted and optimized according to the timbre similarity until it is confirmed that the timbre similarity is greater than a preset similarity threshold.

[0079] In this embodiment, after the target video is generated, the preset sound sample and the target video are further analyzed for timbre similarity. For example, if the insurance agent Zhang San wants to use his own voice in his insurance product introduction video, the system obtains Zhang San's voice sample from the database, and then performs similarity analysis on it with the voice generated in the target video to obtain the timbre similarity between the audio in the target video and Zhang San's voice sample. Specifically, the voice features of the preset sound sample and the audio in the target video are extracted respectively to generate corresponding embeddings, and then the extracted embeddings are statistically modeled, for example, the distribution of these features is described by Gaussian distribution, and then the Fréchet distance between the two Gaussian distributions is calculated by the FAD formula to obtain the corresponding FAD score, and the similarity between the generated audio and the original sound sample is evaluated by the FAD score. The lower the FAD score, the more similar the generated audio is to the original sample, which is converted into the corresponding timbre similarity.

[0080] If the timbre similarity obtained by the current analysis is low, for example, the voice in the target video is not similar enough to Zhang San's voice sample, the voice in the target video is adjusted, such as regenerating audio, pitch, volume, rhythm and other parameters, until the timbre similarity reaches or exceeds the preset threshold, ensuring that the voice in the target video has a high similarity with the preset sound sample, making the video sound more realistic, and improving the quality of the generated video.

[0081] It should be noted that there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0082] Further references Figure 6 , as a response to the above Figure 2 The present invention provides an embodiment of a video generation device based on a video script, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0083] like Figure 6 As shown, the video script-based video generating device 60 described in this embodiment includes:

[0084] A scene confirmation module 601 is used to confirm the application scene of the current video generation in response to the scene selection instruction;

[0085] The script acquisition module 602 is used to acquire the corresponding video script according to the application scenario and the preset acquisition strategy;

[0086] A text analysis module 603 is used to perform text analysis on the video script to extract the text content and auxiliary information of the video script;

[0087] The video generation module 604 is used to obtain a preset sound sample, and generate a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script.

[0088] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions, which is more suitable for describing the video generation execution process based on video scripts than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0089] In one embodiment, the scene confirmation module 601 includes:

[0090] A scene selection unit, used to display a video generation interface, wherein the video generation interface includes a plurality of application scene options;

[0091] The scene confirmation unit is used to receive a scene selection operation of the user on the plurality of application scenes, and confirm the application scene of the current video generation, wherein the application scene is a new video generation scene or a general video multiplexing scene.

[0092] In one embodiment, when the application scenario is a new video generation scenario, the script acquisition module 602 is specifically used to:

[0093] Receive the script file uploaded by the user, perform text extraction processing on the script file, and obtain the corresponding video script.

[0094] In one embodiment, the script acquisition module 602 includes:

[0095] A historical video display unit, used to display a video selection interface, wherein the video selection interface includes a plurality of historically generated general videos;

[0096] A multiplexed video selection unit, used to receive a video selection operation of a user on a plurality of common videos and confirm the video to be multiplexed;

[0097] The video content extraction unit is used to extract the video content of the video to be multiplexed and generate a text description of the video to be multiplexed as a video script of the current application scenario.

[0098] In one embodiment, the video generation module 604 includes:

[0099] A sample acquisition unit, used to acquire a corresponding audio clip from a database as a preset sound sample according to the user's identity identifier;

[0100] An audio generation unit, configured to input the preset sound sample and the text content and auxiliary information of the video script into a trained text-to-speech generation model, extract the timbre features of the preset sound sample, and generate a target audio with corresponding timbre and pitch based on the text content and auxiliary information;

[0101] The video generation unit is used to input the target audio and the text content of the video script into a trained video generation model, generate a corresponding video screen according to the text content, and then synthesize it with the target audio to generate a corresponding target video.

[0102] In one embodiment, the device 60 further includes:

[0103] A preview module is used to display a video preview interface, wherein the video corpus interface includes a currently generated target video;

[0104] A feedback receiving module, used to play the currently generated target video when receiving a preview operation input by the user, and receive feedback adjustment information input by the user;

[0105] The feedback adjustment module is used to adjust the speech rate and / or intonation of the target video according to the feedback adjustment information to generate an adjusted target video.

[0106] In one embodiment, the device 60 further includes:

[0107] A similarity analysis module is used to perform similarity analysis on the preset sound sample and the target video to obtain the timbre similarity between the target video and the preset sound sample;

[0108] The video optimization module is used to adjust and optimize the target video according to the timbre similarity until it is confirmed that the timbre similarity is greater than a preset similarity threshold.

[0109] In the above embodiment, the present invention discloses a video generation device based on a video script, which confirms the application scenario of the current video generation by responding to a scene selection instruction; obtains the corresponding video script according to the application scenario according to a preset acquisition strategy; performs text analysis processing on the video script to extract the text content and auxiliary information of the video script; obtains a preset sound sample, and generates a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script. Through AIGC technology, video scripts can be automatically acquired and generated according to preset scene instructions, which greatly improves the efficiency of content creation, and video content matching the preset sound sample can be generated by using the text content and auxiliary information of the video script, realizing highly personalized and customized video generation, meeting the unique needs of different users, not only reducing manual participation and reducing generation costs, but also generating a large amount of content in a short time while maintaining the consistency of the theme, meeting the needs of rapid production, and improving video generation efficiency.

[0110] Another embodiment of the present invention provides a computer device, such as Figure 7 As shown, the computer device 70 includes:

[0111] One or more processors 701 and memory 702, Figure 7 A processor 701 is used as an example for description. The processor 701 and the memory 702 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0112] Processor 701 is used to complete various control logics of computer device 70, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component or any combination of these components. In addition, processor 701 can also be any traditional processor, microprocessor or state machine. Processor 701 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0113] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the video generation method based on the video script in the embodiment of the present invention. The processor 701 executes various functional applications and data processing of the computer device 70 by running the non-volatile software programs, instructions and units stored in the memory 702, that is, implementing the video generation method based on the video script in the above method embodiment.

[0114] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device 70, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the computer device 70 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 702, and when executed by one or more processors 701, the steps of the video generation method based on the video script in any of the above-mentioned method embodiments are executed.

[0115] In the above embodiment, the present invention discloses a computer device, which confirms the application scenario of the current video generation by responding to the scene selection instruction; obtains the corresponding video script according to the preset acquisition strategy according to the application scenario; performs text analysis processing on the video script to extract the text content and auxiliary information of the video script; obtains the preset sound sample, and generates a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script. Through AIGC technology, the video script can be automatically acquired and generated according to the preset scene instruction, which greatly improves the efficiency of content creation, and the video content and auxiliary information of the video script can be used to generate video content matching the preset sound sample, realizing highly personalized and customized video generation, meeting the unique needs of different users, not only reducing manual participation and reducing generation costs, but also generating a large amount of content in a short time while maintaining the consistency of the theme, meeting the needs of rapid production, and improving the efficiency of video generation.

[0116] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the video generation method based on the video script in any of the above method embodiments are executed.

[0117] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which confirms the application scenario of the current video generation by responding to the scene selection instruction; obtains the corresponding video script according to the application scenario according to the preset acquisition strategy; performs text analysis processing on the video script to extract the text content and auxiliary information of the video script; obtains the preset sound sample, and generates a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script. Through AIGC technology, the video script can be automatically acquired and generated according to the preset scene instruction, which greatly improves the efficiency of content creation, and the video content and auxiliary information of the video script can be used to generate video content matching the preset sound sample, realizing highly personalized and customized video generation, meeting the unique needs of different users, not only reducing manual participation and reducing generation costs, but also generating a large amount of content in a short time while maintaining the consistency of the theme, meeting the needs of rapid production, and improving the efficiency of video generation.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0119] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0120] In summary, the present invention discloses a video generation method, device, equipment and medium based on a video script, wherein the method includes: confirming the application scenario of the current video generation in response to a scene selection instruction; obtaining the corresponding video script according to the application scenario according to a preset acquisition strategy; performing text analysis on the video script to extract the text content and auxiliary information of the video script; obtaining a preset sound sample, and generating a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script. Through AIGC technology, video scripts can be automatically acquired and generated according to preset scene instructions, which greatly improves the efficiency of content creation, and video content matching the preset sound sample can be generated by using the text content and auxiliary information of the video script, realizing highly personalized and customized video generation, meeting the unique needs of different users, not only reducing manual participation and reducing generation costs, but also generating a large amount of content in a short time while maintaining the consistency of the theme, meeting the needs of rapid production, and improving video generation efficiency.

[0121] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0122] It should be noted that if any software tools or components other than those of the Company appear in the embodiments of the present application, they are merely used for illustration and do not represent actual use. It should be understood that the application of the present invention is not limited to the above examples, and that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes shall fall within the scope of protection of the claims attached to the present invention.

Claims

1. A video generation method based on a video script, characterized in that: include: In response to the scene selection instruction, confirming the application scene of the current video generation; Acquire the corresponding video script according to the application scenario and the preset acquisition strategy; Performing text analysis on the video script to extract text content and auxiliary information of the video script; A preset sound sample is obtained, and a target video matching the preset sound sample and the video script is generated according to the preset sound sample and the text content and auxiliary information of the video script.

2. The video generation method based on video script according to claim 1, characterized in that: The step of confirming the application scenario of the current video generation in response to the scene selection instruction includes: Displaying a video generation interface, wherein the video generation interface includes several application scenario options; A scene selection operation of the user on the plurality of application scenes is received, and an application scene for current video generation is confirmed, where the application scene is a new video generation scene or a general video multiplexing scene.

3. The video generation method based on video script according to claim 2, characterized in that: When the application scenario is a new video generation scenario, obtaining a corresponding video script according to a preset acquisition strategy based on the application scenario specifically includes: Receive the script file uploaded by the user, perform text extraction processing on the script file, and obtain the corresponding video script.

4. The video generation method based on video script according to claim 2, characterized in that: When the application scenario is a general video multiplexing scenario, obtaining the corresponding video script according to a preset acquisition strategy based on the application scenario specifically includes: Displaying a video selection interface, wherein the video selection interface includes a plurality of historically generated general videos; Receive a video selection operation of a user for a plurality of general videos, and confirm the video to be multiplexed; Video content is extracted from the video to be multiplexed, and a text description of the video to be multiplexed is generated as a video script of the current application scenario.

5. The video generation method based on video script according to claim 1, characterized in that: The step of obtaining a preset sound sample and generating a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script comprises: According to the user's identity, a corresponding audio clip is obtained from a database as a preset sound sample; Inputting the preset sound sample and the text content and auxiliary information of the video script into a trained text-to-speech generation model, extracting the timbre features of the preset sound sample, and generating a target audio with corresponding timbre and pitch based on the text content and auxiliary information; The target audio and the text content of the video script are input into a trained video generation model, and the corresponding video screen is generated according to the text content and then synthesized with the target audio to generate the corresponding target video.

6. The video generation method based on video script according to claim 1, characterized in that: After obtaining the preset sound sample and generating a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script, the method further includes: Displaying a video preview interface, wherein the video corpus interface includes the currently generated target video; When receiving a preview operation input by the user, playing the currently generated target video, and receiving feedback adjustment information input by the user; The speech rate and / or intonation of the target video is adjusted according to the feedback adjustment information to generate an adjusted target video.

7. The video generation method based on video script according to claim 1, characterized in that: After obtaining the preset sound sample and generating a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script, the method further includes: Performing similarity analysis on the preset sound sample and the target video to obtain the timbre similarity between the target video and the preset sound sample; The target video is adjusted and optimized according to the timbre similarity until it is confirmed that the timbre similarity is greater than a preset similarity threshold.

8. A video generation device based on a video script, characterized in that: include: A scene confirmation module, used to confirm the application scene generated by the current video in response to the scene selection instruction; A script acquisition module, used to acquire the corresponding video script according to the application scenario and a preset acquisition strategy; A text analysis module, used to perform text analysis on the video script and extract text content and auxiliary information of the video script; The video generation module is used to obtain a preset sound sample, and generate a target video matching the preset sound sample and the video script according to the preset sound sample and the text content and auxiliary information of the video script.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video generation method based on the video script according to any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the video generation method based on the video script as described in any one of claims 1-7.

Images