Automatic video generation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses an automatic video generation method, which is characterized in that a video generation material and a content text corresponding to a video theme are obtained by determining the video theme, and the video generation material can be a video or a picture. The content text is converted into voice data through the text-to-voice model, voice materials are generated, and the voice materials are used for generating video clips. And generating a synchronous subtitle of each voice material, embedding the subtitles into the video clips, and synchronously displaying the subtitles and the voice materials. And generating independent video clips according to the video generation material, the voice material and the synchronous subtitles, and generating a complete video by combining the independent video clips. According to the method and the device, a mode of automatically processing a plurality of materials and content texts is adopted, so that the requirement of manual intervention is greatly reduced, the video generation efficiency is optimized, and common time consumption and labor cost in a manual video production process are avoided.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to an automatic video generation method, device, equipment and storage medium. Background Art

[0002] In the financial sector, the generation and application of video content has gradually become an important means to improve user experience, promote financial products, and conduct education and training. Video generation covers a wide range of scenarios from customer service to marketing promotion. As demand grows, the efficiency and quality of video generation are becoming increasingly prominent. At present, video generation technology faces the following deficiencies:

[0003] At present, most financial institutions still rely on manual production for video content. Whether it is a promotional video, training course, or product introduction video, professional personnel are required to shoot, edit, and post-produce. This process is not only time-consuming and costly, but also difficult to automate on a large scale. In the financial industry, content updates quickly and the demand for videos is high. The manual video production model cannot meet the needs of quickly generating high-quality videos.

[0004] Although AI technology has made some progress in other fields, in the field of video generation, the quality of AI-generated videos is often unsatisfactory. Problems such as video quality, smoothness, and synchronization of subtitles and voices frequently occur. Especially when it comes to complex financial scenarios, the generated videos cannot compare with manually produced videos. These technical limitations make the automatically generated videos have obvious gaps in visual effects, content expression, and user experience, and cannot effectively replace manually produced videos.

[0005] From the selection of video materials, the conversion of text content to the rendering of the final video, the process of video generation is cumbersome and involves many steps. Most existing automation tools can only handle a part of the process and it is difficult to automate the entire process. Especially in the financial industry, which involves a large amount of data and sophisticated information processing, video generation often requires multiple manual interventions, further increasing the complexity and cost of the operation.

[0006] In summary, the current application of automatic video generation technology in the financial field still has many shortcomings, including high reliance on manual labor, unstable generation quality, and difficulty in achieving full process automation. In order to solve these problems, it is urgent to develop smarter and more efficient automatic video generation technology to meet the growing and diverse video needs of the financial industry. Summary of the invention

[0007] The main purpose of the present invention is to provide an automated video generation method, device, equipment and storage medium, aiming to solve the technical problem in the prior art that video generation is highly dependent on manual labor and it is difficult to achieve efficient integration of multiple materials and full-process automation.

[0008] To achieve the above object, the present invention provides an automatic video generation method, comprising:

[0009] Determine a video theme, and obtain a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials;

[0010] Convert each content text into speech data through a text-to-speech model to generate corresponding speech material, which is used to generate video clips;

[0011] Generate corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material;

[0012] Generate independent video clips based on each video generation material and corresponding voice material and synchronized subtitles;

[0013] Merge all independent video clips into a complete video to obtain the target video.

[0014] Furthermore, to achieve the above object, the present invention provides an automatic video generation device, comprising:

[0015] A video generation material acquisition module, used to determine a video theme, and acquire a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials;

[0016] A voice material generation module, used to convert each content text into voice data through a text-to-speech model, and generate corresponding voice materials, which are used to generate video clips;

[0017] A subtitle generation and synchronization module, used to generate corresponding synchronized subtitles based on each voice material, and the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material;

[0018] A video segment generation module, used to generate independent video segments based on each video generation material and corresponding voice material and synchronized subtitles;

[0019] The video synthesis module is used to merge all independent video clips into a complete video to obtain the target video.

[0020] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and an automatic video generation program stored in the memory and executable on the processor, and the automatic video generation program, when executed by the processor, implements the steps of the automatic video generation method as described above.

[0021] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which an automatic video generation program is stored, and when the automatic video generation program is executed by a processor, the steps of the automatic video generation method as described above are implemented.

[0022] Beneficial effects: The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses an automated video generation method. By determining the video theme, video generation materials and content texts corresponding to the video theme are obtained, and the video generation materials can be videos or pictures. The content text is converted into voice data through a text-to-speech model to generate voice materials, which are used to generate video clips. Synchronous subtitles are generated for each voice material, and the subtitles are embedded in the video clip and displayed synchronously with the voice material. Independent video clips are generated according to the video generation material, voice material and synchronized subtitles, and a complete video is generated by merging independent video clips. The present invention greatly reduces the need for manual intervention by adopting an automated processing method for multiple materials and content texts, optimizes the efficiency of video generation, and avoids the time consumption and labor costs commonly seen in the process of manually making videos. The problem of high labor cost and insufficient quality of automated processing in existing video generation technologies is solved, ensuring that the generated video has high quality and the effect of voice and subtitle synchronization, realizing full-process automated processing, and meeting the needs of large-scale, efficient and accurate video generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0024] Figure 1 A schematic diagram of an application environment of an automatic video generation method in an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a flow chart of an embodiment of an automatic video generation method of the present invention;

[0026] Figure 3 A schematic diagram of functional modules of a preferred embodiment of the automatic video generation device of the present invention;

[0027] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0028] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0030] The automatic video generation method provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the user end communicates with the server end through a network. The server end can determine the video theme through the user end, obtain the video generation material and content text corresponding to the video theme, and the video generation material can be a video or a picture. The content text is converted into voice data through a text-to-speech model to generate voice material, and the voice material is used to generate video clips. Generate synchronized subtitles for each voice material, and the subtitles are embedded in the video clip and displayed synchronously with the voice material. Generate independent video clips according to the video generation material, voice material and synchronized subtitles, and generate a complete video by merging independent video clips. The present invention greatly reduces the need for manual intervention by adopting an automated processing method for multiple materials and content texts, optimizes the video generation efficiency, and avoids the time consumption and labor costs commonly seen in the manual video production process. The problem of high labor cost and insufficient automated processing quality in the existing video generation technology is solved, ensuring that the generated video has high quality and the effect of voice and subtitle synchronization, realizing full-process automated processing, and meeting the large-scale, efficient and accurate video generation requirements. Among them, the user end can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server end can be implemented with an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0031] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of the automatic video generation method provided by the present invention. It should be noted that although the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0032] like Figure 2 As shown, the automatic video generation method proposed by the present invention comprises the following steps:

[0033] S10, determining a video theme, acquiring a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials;

[0034] In this embodiment, the video theme is the core of the entire video generation process and determines the overall direction of the video content. The video theme can be generated based on keywords input by the user, preset templates, or automatically analyzed by the system, and the analysis method includes analysis of historical data or semantic recognition. By identifying the user's needs or scenarios, the system can generate a video theme that meets specific business goals.

[0035] The determination of video topics can be done through keyword matching, semantic analysis tools, or user selection. In the financial field, the corresponding topics can be generated based on keywords entered by customers, such as "insurance product introduction" or "financial security training." The system can automatically select relevant topics from preset templates, or generate topics that meet the needs from historical data based on semantic analysis technology.

[0036] Video generation materials include video clips and image materials, which are the basic elements of video generation. The acquisition process of materials can be retrieved from local libraries, online resource libraries (such as material websites or cloud databases). The selection of materials must be highly matched with the theme of the video, which is usually achieved by analyzing the tags, keywords and scene classification information of the video materials.

[0037] The system will automatically call up resources in the material library and search based on the keywords or classification tags of the video theme. In the financial data application scenario, the system can retrieve relevant video or picture materials such as "bank service process" and "financial product promotion" from the material library to ensure that the generated video content is consistent with the financial theme.

[0038] Content text is the basis for generating voice materials and subtitles, and is closely related to video materials. Content text can be automatically generated by a large language model, or it can be a descriptive text provided by the user. The generation and selection of content text should ensure its accuracy and professionalism, especially in the financial field, where the text must meet professional standards to avoid misleading information.

[0039] The system can automatically generate text content that matches the video material, using a pre-trained language model (such as GPT). At the same time, users can also upload specific text content. In the financial data scenario, the content text may include a description of a financial product, market data analysis, or an introduction to a risk control strategy.

[0040] The generation of video clips depends on video materials and content text. Each video clip contains corresponding materials and text content, and is combined according to the theme. The generated video clips should ensure the coherence of the content, and the voice materials and subtitles should be displayed synchronously with the video screen.

[0041] The system automatically generates video clips based on video materials, voice materials and subtitles. In the financial field, it can generate video clips such as "Interpretation of Financial Policies", which display financial data, charts and text content, and explain the specific content through voice narration.

[0042] Example description: In the financial industry, video generation technology can be used to produce financial product introduction videos, market analysis reports or online training courses. For example, based on the theme of "insurance product introduction", the system automatically obtains relevant video materials and texts, generates complete video clips, displays the characteristics and advantages of insurance products, and combines data charts for risk control analysis. In this way, financial institutions can quickly generate high-quality video content for customer promotion or internal training.

[0043] By combining video generation materials and content text in an automated way, the cost and time of manual video production are greatly reduced. At the same time, the accuracy and professionalism of the generated video content are ensured. Especially in the financial field, videos that meet business needs can be quickly generated, effectively improving the efficiency and quality of video generation.

[0044] S20, converting each content text into voice data through a text-to-speech model to generate corresponding voice material, where the voice material is used to generate the video clip;

[0045] In this embodiment, the content text is converted into voice data through a text-to-speech (TTS) model, which is a key step in converting written content into audio form. The TTS model is based on natural language processing technology and can generate natural and fluent speech based on the input text, ensuring that the intonation, speed and emotional expression of the speech match the text content.

[0046] The system will call the pre-trained TTS model to convert the input content text into voice data. In the financial field, if the content text is "Investment Risk Analysis Report", the TTS model will convert this text into voice and generate natural-sounding commentary for subsequent video clip generation. You can use mainstream TTS technologies, such as Google TTS, Microsoft Azure TTS, etc., to achieve high-quality voice generation.

[0047] Voice material is an important element when generating video clips, which directly affects the presentation effect of the video. The generated voice material should be consistent with the content text, and ensure that the voice is clear and accurate to meet the usage requirements of the video clip.

[0048] The system will compare the voice data generated by TTS with the content text to ensure that the voice and text are completely consistent and avoid content deviation. For the application scenarios of financial data, the generated voice materials can be used as financial product explanations, market analysis instructions, etc., for the generation of subsequent video clips.

[0049] As an integral part of a video clip, voice material is used synchronously with video material and subtitles. Voice material is usually used in a video clip to explain or interpret the content of the video, helping the audience to better understand the video content.

[0050] In the generated video clips, the system uses voice material and video material synchronously. Taking financial data as an example, when visual elements such as charts and market analysis are displayed in the video clips, the voice material serves as background commentary to provide detailed explanations for the audience, making the content more organized and logical.

[0051] Example description: In the application scenario of the financial industry, suppose a bank is making a video of an investment risk analysis report. In this process, the system first converts the written investment risk analysis report content text into voice data through a text-to-speech (TTS) model.

[0052] The content text includes an interpretation of the current market risks, such as "The market has been volatile recently, and investors should pay attention to risk diversification and avoid excessive concentration in a single investment product." The system calls the TTS model to convert the above content into natural speech. The generation of speech not only includes pronunciation, but also adjusts the tone according to the emotional expression in the text, such as using a more cautious tone when mentioning "risk." The generated voice material will be used as the commentary part of the video clip. For example, when the video shows the current stock market fluctuations, the voice material will simultaneously explain the specific details of the market risk analysis to help the audience better understand the market environment.

[0053] Automatically generating speech materials through the text-to-speech model reduces the time and cost of manually recording speech. It ensures the consistency between speech and text and improves the efficiency of speech generation, which is especially suitable for large-scale video generation needs. In the financial field, automatically generating speech materials can quickly convert professional content into commentary audio, enhancing the professionalism and practicality of videos.

[0054] S30, generating corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material;

[0055] In this embodiment, the generated voice material is compared with the corresponding content text, and synchronized subtitles are generated according to the timeline of the voice. Each segment of the voice material has a specific start and end time, and the subtitles need to accurately correspond to the pronunciation content of the voice. Synchronized subtitles are not only about matching text with voice, but also need to consider the display duration of the subtitles and the smoothness of video playback.

[0056] The system analyzes the timeline of the voice material, identifies the duration and pronunciation rhythm of each voice segment, and generates subtitles that are synchronized with it based on this information. Subtitles can be generated by segmenting the content text to ensure that the display of subtitles is synchronized with the playback of the voice segment. In the financial field, such as in the "Financial Product Introduction" video, the system will generate corresponding subtitles for each voice segment and make them consistent with the length of the commentary.

[0057] After the synchronized subtitles are generated, they need to be embedded into the video clips and displayed synchronously with the audio material. This step requires that the subtitles appear at the same time as the video content and be synchronized with the audio material. It is necessary to ensure not only the correctness of the subtitle content, but also the readability and visual effect of the subtitles.

[0058] The system will embed the generated synchronized subtitles into the audio track timeline of the video, so that the subtitles appear at the same time as the voice material. The system will configure the font, color, size and display position of the subtitles to ensure that the subtitles are readable in the video clip and do not interfere with the picture. In the application of the financial field, such as when explaining the "market risk analysis" video, the system will embed the generated subtitles and the commentary voice into the video synchronously to help the audience better understand the analysis content.

[0059] Example: Assuming a financial institution produces a video on "Introduction to Investment Products", the system first generates synchronized subtitles from the voice material of the narration and ensures that the subtitles exactly match the timing of each voice segment. When the video clips show the specific features of the product, the subtitles synchronously display the product's name, features, and market analysis, so that the audience can not only hear the narration, but also intuitively obtain key information through the subtitles. This way of embedding synchronized subtitles helps the audience quickly and accurately understand the key points of financial products.

[0060] By automatically generating synchronized subtitles and embedding them with the voice material, the time and cost of manually editing subtitles are reduced. This ensures that the subtitles are fully synchronized with the voice content and improves the readability and professionalism of the video. In the financial field, the generated subtitles can accurately convey financial terms and analytical content, improving the user experience and content understanding of the video.

[0061] S40, generating an independent video segment based on each video generation material and the corresponding voice material and synchronized subtitles;

[0062] In this embodiment, the video generation material includes video clips or picture materials, which are the visual content basis of the video clip. The voice material is generated by the text-to-speech model and matched with the content text. The synchronized subtitles are text content generated based on the voice material to assist in understanding the explanation in the video. By combining these elements, a complete video clip containing video, voice and subtitles is generated.

[0063] The system will match the voice material with the timeline of the video-generated material and generate synchronized subtitles. The video-generated material can be a video clip, picture or animation, the voice material is used to explain or explain the content in the video material, and the subtitles are used to assist the audience to understand the voice and video content. For example, in the financial industry, the system can use data charts as video materials, while using voice to explain market trends and further explain specific data points through subtitles.

[0064] An independent video clip is a complete video unit that combines video generation materials, audio materials, and synchronized subtitles. Each video clip is an independent and fully structured content block that can be displayed alone or combined with other video clips to form a complete video. The process of generating independent video clips needs to ensure the synchronous display of video, audio, and subtitles, and render and export according to the goals of video generation.

[0065] The system will synthesize the corresponding voice material and subtitles according to the playback time of each video material, and generate an independent video clip containing all elements. In the financial field, independent video clips may include "interpretation of financial reports" and "market analysis briefings". The system will create independent audio and video tracks for each clip, and ensure that the subtitles are displayed synchronously with the voice, and finally render independent video clips for subsequent use or merging.

[0066] Example description: When making a video about "Financial Market Quarterly Report", the system first uses market data charts as video generation materials, and combines them with market analysis voice materials to generate synchronized subtitles. The system combines these materials to generate independent video clips, each of which contains an interpretation of market performance or investment advice for a certain quarter. The generated independent video clips can be played alone or combined with other clips to form a complete quarterly report video for financial institutions to use for internal training or customer presentations.

[0067] By combining video generation materials, voice materials and synchronized subtitles to generate independent video clips, the modular generation of video content is achieved, which improves the flexibility and efficiency of video production. It reduces the time of manual intervention while ensuring the unity of video content in terms of vision, hearing and text explanation. In the financial field, through the generation of independent video clips, financial institutions can quickly produce multiple modular video contents for product promotion, market analysis or customer education.

[0068] S50, merging all independent video clips into a complete video to obtain a target video.

[0069] In this embodiment, after generating multiple independent video clips, these clips need to be merged in a logical order to finally generate a complete video file. Each video clip includes video generation material, voice material and synchronized subtitles. When merging, it is necessary to ensure that the transition effect between each clip is natural and smooth. Usually, the clip merging process also involves the smooth connection of audio tracks and the processing of video switching transition effects.

[0070] The system first arranges independent video clips according to a predetermined order (such as timeline, content logic). During the merging process, the system automatically processes the audio and video transition effects between clips to ensure that the audio track of each video clip is seamlessly connected, and uses video transition effects such as fade in and fade out, switching, etc. to make the video playback natural and smooth. In the application scenarios in the financial field, such as the production of an "annual financial report", independent video clips of market performance in multiple quarters will be merged into a complete annual summary report.

[0071] After merging all the independent video clips, the generated target video is a complete video file containing all the content. This complete video not only contains the video material, voice material and synchronized subtitles, but also maintains the logical order of each independent clip. The complete video should meet the requirements of the output format for further use or sharing.

[0072] After merging the independent video clips, the system performs the final video rendering and generates a target video containing all the merged content. The video can be exported to multiple formats, such as MP4, AVI, etc., for use in different playback environments. Financial institutions may need to generate a "comprehensive market analysis video" that integrates market reports from multiple quarters and merges them into a complete annual analysis report for easy display and sharing.

[0073] Example: When making an "Annual Financial Market Analysis Report" video, the system first generates four independent quarterly market analysis video clips. Through the automatic merging function, the system merges these four clips into a complete annual analysis report in chronological order. During the merging process, the system automatically applies a fade-in and fade-out transition effect to ensure smooth video playback. The generated target video shows the market changes and investment strategy summary for the whole year, which is used by financial institutions for annual report display and sharing.

[0074] By merging multiple independent video clips into a complete video, the flexibility and efficiency of video generation are improved. It not only reduces the time of manual editing, but also ensures the logical coherence and content consistency of the video, and the final generated target video has a better viewing effect. In the financial field, the merged video helps to quickly integrate content from multiple time periods and multiple topics, providing users with more comprehensive market analysis or product introductions, meeting the needs of efficient content display.

[0075] The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses an automated video generation method. By determining a video theme, a video generation material and a content text corresponding to the video theme are obtained, and the video generation material can be a video or a picture. The content text is converted into voice data through a text-to-speech model to generate voice material, and the voice material is used to generate a video clip. Synchronous subtitles are generated for each voice material, and the subtitles are embedded in the video clip and displayed synchronously with the voice material. Independent video clips are generated according to the video generation material, the voice material and the synchronized subtitles, and a complete video is generated by merging the independent video clips. The present invention greatly reduces the need for manual intervention by adopting an automated processing method for multiple materials and content texts, optimizes the video generation efficiency, and avoids the time consumption and labor costs commonly seen in the manual video production process. The problem of high labor cost and insufficient automated processing quality in the existing video generation technology is solved, and it is ensured that the generated video has high quality and the effect of voice and subtitle synchronization, and the full process is automated. Processing is realized, and the needs of large-scale, efficient and accurate video generation are met.

[0076] In one embodiment, the above S40 includes:

[0077] S401, if the video generation material is a video material, use a large language model to summarize the content text corresponding to the video generation material to form key information;

[0078] S402, based on the key information, using video moment retrieval technology to search and extract key video clips in the video generation material;

[0079] S403, processing the extracted key video clips to remove the original sound in the key video clips;

[0080] S404, synchronously embedding the voice material and the synchronized subtitles into the processed key video clip;

[0081] S405, using a video processing tool to render the processed key video clip to generate an independent video clip including video material, voice material and synchronized subtitles.

[0082] In this embodiment, the large language model is used to analyze the content text corresponding to the video generation material and extract the most core information from it. The key information is the basis for subsequent video segment selection and processing, and is used to determine the most important part or segment in the video.

[0083] The system uses a pre-trained large language model (such as GPT) to perform semantic analysis on the content text corresponding to the video material and automatically generate a summary of key information. Key information may include the theme, key points or key scenes to be conveyed in a video. For example, in the "market trend analysis" video material in the financial field, the system can extract key information such as "quarterly market return rate" or "risk control strategy."

[0084] Video moment retrieval technology is used to find segments related to key information from video generation materials. This technology quickly locates the key parts of the video by analyzing the timeline, visual content and semantic information in the video.

[0085] Based on the key information generated by the large language model, the system uses the video moment retrieval algorithm to find and extract the most relevant clips in the video material. The retrieval method can be based on timeline, video frame key points, or visual object detection. For example, in a financial data interpretation video, the system can extract key clips that show "market fluctuations" or "investment advice."

[0086] In the extracted key video clips, the original audio is removed to avoid interfering with the newly generated voice commentary. This step helps to align the new voice material with the video content and prevent audio overlap.

[0087] The system will automatically delete the original audio part of the extracted key video clips, retain the video content, and ensure that the audio track is blank for subsequent embedding of new voice materials. In the financial industry, if the video material contains outdated or irrelevant commentary, the system can automatically remove these audio parts.

[0088] In the processed key video clips, voice materials and synchronized subtitles are embedded as commentary and auxiliary reading parts to ensure the content consistency of video, audio and subtitles.

[0089] The system will embed the generated voice material and synchronized subtitles into the processed key video clips. The embedding process ensures that the voice and subtitles are played synchronously with the video clips, allowing users to hear the commentary and see the subtitles at the same time. In financial data applications, the generated voice material may be used to explain market trends, while the subtitles display key information, such as important market data or investment advice.

[0090] Rendering is the process of integrating all elements (video material, voice material and subtitles) into a playable independent video clip. The rendering process ensures that the video plays smoothly and the elements are synchronized.

[0091] The system calls the video processing tool to render the processed key video clips and generate complete independent video clips containing video materials, voice materials and synchronized subtitles. The rendered clips can be exported to common video formats for subsequent playback or editing. For example, in the financial field, independent video clips containing market report interpretation, data display and subtitles can be generated for conference presentations or customer sharing.

[0092] This embodiment improves the efficiency of video generation and reduces the time of manual editing and adjustment by automatically generating and merging video materials, voice materials and synchronized subtitles. It ensures the accuracy and consistency of video content, especially in financial applications, and can quickly generate high-quality video content containing financial data analysis and interpretation, meeting the financial industry's needs for data presentation, analysis and interpretation.

[0093] In one embodiment, the above S40 includes:

[0094] S406, if the video generation material is a picture material, use a large language model to summarize the content text corresponding to the video generation material to form key information;

[0095] S407, searching the video generation material using a picture-text matching model to determine a key picture matching the key information;

[0096] S408, generating a continuous dynamic video segment from the key image through a transition special effect;

[0097] S409, synchronously embedding the voice material and the synchronized subtitles into the dynamic video segment;

[0098] S410, rendering the dynamic video segment to generate an independent video segment including picture materials, voice materials and synchronized subtitles.

[0099] In this embodiment, the large language model extracts the core information of the text, i.e., key information, by analyzing the content text related to the picture material. This key information is used to determine the key parts of the picture material or the content related to video generation, and helps the subsequent picture-text matching and video generation.

[0100] The system will call a large language model (such as GPT) to analyze the description text of the image material and summarize the most important information. For example, in the financial field, if the content text is about "stock market volatility analysis", the system will extract key points such as "stock volatility" and "risk analysis" as key information for subsequent image material selection and processing.

[0101] The image-text matching model is used to match the summarized key information with the image material to find the key image that is most relevant to the text information. The model determines whether the image matches the key information by analyzing the visual features of the image and the semantic features of the text.

[0102] The system uses a picture-text matching model to extract features from pictures and perform semantic comparisons with text to determine the picture that best matches the key information. For example, if the content text mentions "stock market trends," the system will find pictures showing stock market curves in the picture material as key pictures. In the financial industry, this matching can help quickly find visual materials for displaying market analysis.

[0103] After determining the key images, the system generates continuous dynamic video segments from these static images through transition effects. These transition effects can include fade-in, fade-out, sliding, zooming, etc., with the purpose of making the transition between images more natural and forming smooth video content.

[0104] The system will select appropriate transition effects (such as fade in, fade out, wipe or move) to string together key images into a continuous video clip. For example, in the financial field, multiple market analysis chart images can be used to generate a video segment through sliding transitions to show data changes and market trends.

[0105] Voice materials and synchronized subtitles are used to explain and assist in understanding the content displayed in the pictures. The generated voice materials and subtitles must be embedded in the video segments synchronously with the picture content, so that the audience can hear the relevant explanations and see the subtitles while watching the picture changes.

[0106] The system will embed the generated voice material and synchronized subtitles into the audio and subtitle tracks of the dynamic video segment. In financial applications, such as the video segment of "Market Risk Analysis", the voice material will interpret the displayed charts, while the subtitles will display key information, such as risk assessment indicators, to help viewers understand the video content.

[0107] Rendering is the process of integrating all elements (images, audio, and subtitles) into a complete playable video clip. The rendering process ensures that the dynamic video clips are played smoothly, and that the audio and subtitles are displayed synchronously with the screen.

[0108] The system calls the video processing tool to render the dynamic video segments and generate complete independent video clips. The rendered video can be exported to common video formats (such as MP4, etc.) for subsequent playback and use. In financial applications, the generated video clips can be used to display key data charts in quarterly reports and provide detailed analysis through voice commentary.

[0109] Example description: Assume that a financial institution needs to produce a "quarterly market analysis report" video. The system first extracts key information such as "stock market performance" and "risk assessment" based on the content text. Next, the system uses the image-text matching model to select key images showing "stock curve chart" and "market risk chart" from the material library. Through transition effects, these images are combined into a dynamic video segment to show changes in market data. Then, the system embeds voice material into the video segment, uses commentary to introduce market performance, and displays key information through subtitles. Finally, the system renders the video segment to generate a complete video clip containing image material, voice material, and synchronized subtitles for displaying the quarterly report.

[0110] This embodiment automatically generates dynamic video segments by combining picture materials, voice materials and synchronized subtitles, thereby improving the efficiency and quality of video generation. It can not only convert static pictures into coherent video content, but also ensure that the voice and subtitles are synchronized with the picture display, thereby improving the visual coherence and information transmission efficiency of the video. In the financial field, it can quickly generate video clips such as market analysis and product introduction for internal reporting or customer presentation.

[0111] In one embodiment, the above S40 includes:

[0112] S411, matching video background music according to the video theme;

[0113] S412, determining the volume of the background music of the video according to the clarity of the voice material and the video scene requirements;

[0114] S413, creating an audio track, and synchronously embedding the video background music and the voice material into the audio track;

[0115] S414, rendering the video clip to generate an independent video clip including video background music, video generation material, corresponding voice material and synchronized subtitles.

[0116] In this embodiment, background music matching is performed by analyzing the characteristics of the video theme and selecting music that matches the emotion, scene, content type, etc. of the video.

[0117] The theme of a video usually has an emotional tone, which can be serious, cheerful, calm, etc. By analyzing the emotional characteristics of the video theme, the system can identify the type of background music that is suitable for the theme. For example, a video about financial market analysis usually needs to choose a steady and calm background music, while a product launch video may choose a more active and energetic music.

[0118] The system uses natural language processing (NLP) technology to perform semantic analysis on the text content of the video theme and extract the emotional tone. For example, if the theme is "quarterly financial market analysis", the system extracts labels such as "stable" and "professional", indicating that the music style needs to be stable and not interfere with the commentary content. At this time, the system does not consider other content of the video, and only guides the selection of music based on the theme.

[0119] Each video theme has its own specific emotional tone, and the music should be chosen to align with these emotions. For example, if the video theme is "Annual Market Review", the background music should be more low-key and calm; while for the theme of "Market Outlook", you can choose music that conveys hope or motivation.

[0120] The system compares the preset music library tags with the emotional tone of the theme. For example, the system extracts music with emotional tags such as "stable", "professional", and "low-key" from the music library to match the emotional tone of the video theme. Each track in the music library has an emotional tag (such as motivational, quiet, and cheerful), and the system selects candidate music that meets the requirements through emotional matching.

[0121] According to the emotional tone of the theme, the system retrieves music tracks that match the emotion in the music library. The background music library is usually organized according to dimensions such as emotional tags, tones, and styles. Through the matching process, the system can find music that matches the theme of the video.

[0122] The system matches the emotional tone extracted from the video theme with the emotional tags in the music library and retrieves music that matches these emotions. For example, for the theme of "market review", the system will give priority to music clips with steady emotional tone and smooth melody.

[0123] The volume of background music is related to the clarity, speaking speed, pitch and other factors of the voice material. Background music should not interfere with the clarity of the voice material. The volume should be determined according to the needs of the video scene to balance the audio performance of the background music and the commentary, ensuring that the audience can clearly hear the commentary while enjoying the auxiliary effect of the background music.

[0124] The system analyzes the volume and clarity of the voice material and the atmosphere of the video scene, and automatically adjusts the volume of the background music. The volume is adjusted through an automated algorithm to ensure that the voice material is in a dominant position and the background music volume is low but still present. For example, in a video explaining "Financial Risk Management Strategies", the system will adjust the background music volume to a low level to clearly convey important information.

[0125] Background music and voice materials need to be embedded in the same audio track to ensure that the background music and commentary are played synchronously. The creation of audio tracks involves integrating multiple audio contents into the same timeline so that background music and voice materials can be output at the same time and keep the same rhythm.

[0126] The system will create an audio track, embed the background music and voice material at the same time, and ensure that the playback timeline of the two is consistent. In the financial field, for example, in the video clip of the "quarterly market analysis", the system will create an audio track so that the background music and the commentary are played at the same time, with the commentary part dominating the volume and the background music playing the role of setting the atmosphere.

[0127] Rendering is the process of integrating all elements into a complete video clip, including video generation materials, background music, voice materials and subtitles. The rendering process ensures that the audio and video are synchronized, and the generated independent video clip can be used for playback, sharing or further editing.

[0128] The system will call the video rendering tool to render the video generation material, background music, voice material and subtitles into the same video clip. The generated video clip contains all the content and is finally output in a standard video format, such as MP4. In the financial field, this rendering process is suitable for generating a complete market report video clip, showing the data with background music, commentary and subtitles.

[0129] Example description: When making a video about "Annual Financial Market Performance", the system first selects a relaxing and steady background music based on the theme of the video to suit the content of the market report. Then, the system automatically adjusts the volume of the background music based on the voice material when explaining the market data, so that the explanation is clear and easy to understand without being disturbed by the background music. Then, the system creates an audio track, embeds the background music and the voice material synchronously, and renders a complete annual report video clip. The video not only shows the market data, but also enhances the audience's sense of immersion through the background music, making the financial information transmission more vivid and interesting.

[0130] This embodiment automatically matches the background music according to the video theme and adjusts the volume according to the voice material, so that the generated independent video clips are richer and have better emotional expression effects. The synchronous embedding of background music and voice commentary improves the viewing and professionalism of the video.

[0131] In one embodiment, the above S50 includes:

[0132] S501, sorting all generated independent video clips according to video themes, timelines or scene sequences;

[0133] S502, splicing each independent video segment by using a video editing tool, and processing transition effects between the independent video segments;

[0134] S503, merging the audio tracks of each independent video clip and processing the connection between the voice materials;

[0135] S504, rendering the spliced ​​video and the merged audio track to generate a complete target video.

[0136] In this embodiment, the sorting of video clips is the first step in the merging process, and multiple independent video clips are arranged according to the theme, time sequence or logical sequence of the video scenes. The purpose of sorting is to ensure that the generated target video content structure is clear and conforms to the viewing logic.

[0137] The system will automatically arrange all generated independent video clips according to the subject classification, timeline, or preset scene order when the video is generated. For example, in the financial field, the system can arrange each market analysis video clip in the order of quarterly reports to ensure that the video content is presented in sequence from the first quarter to the fourth quarter to form a complete annual report.

[0138] Video clip splicing refers to merging multiple independent video clips into a continuous video. During the splicing process, the transition effects between clips (such as fade in and fade out, slide, switch) can improve the smoothness of the video and make the video playback more natural.

[0139] The system uses video editing tools to seamlessly stitch together the sorted independent video clips. In order to ensure the visual effect of the clip transition, the system automatically applies transition effects such as fade in and fade out, dissolve, slide, etc., so that the switching between clips is smooth and natural. For example, in a financial video, when the interpretation of market performance for a quarter is completed, the system will add a fade-out effect and then cut into the analysis clip of the next quarter to ensure a smooth transition.

[0140] Merging and processing audio tracks is an important step to ensure the sound consistency of the video. Each independent video clip contains voice material and background music. During the merging process, it is necessary to process the connection of these audios to avoid sudden changes in volume or incoherent voice materials.

[0141] The system will merge the audio tracks of each video clip and process the smooth transition of volume according to the connection between the voice materials to ensure that the voice materials are played continuously without interruption. For example, in a financial analysis video, if a segment of market data ends, the system will smoothly transition to the next segment of commentary to avoid abrupt switching of voice content and ensure the continuity of information transmission.

[0142] Rendering is the process of converting the spliced ​​video clips and the merged audio tracks into a complete video file. The rendering process ensures that the video and audio are played synchronously and generates the final usable video output format.

[0143] The system calls the video rendering tool to unify the spliced ​​video and the merged audio track and output them in a common video format (such as MP4, AVI, etc.). The rendering process ensures the synchronous playback of video, voice material and background music. In the financial field, the final target video may be a complete "Annual Market Analysis Report" video, showing the market trends and risk analysis throughout the year, with commentary voice and background music.

[0144] This embodiment quickly generates a complete video through automatic sorting, splicing, audio merging and rendering, reducing the time and cost of manual editing, ensuring the logical order and consistency of video and audio, and improving the video's viewing experience and information transmission effect.

[0145] In one embodiment, in the above S10, determining a video theme and obtaining a plurality of video generation materials and a plurality of content texts corresponding to the video theme include:

[0146] S101, obtaining a video theme, analyzing the video theme to determine video features, the video features including content type, emotional expression or scene requirements;

[0147] S102, acquiring video generation material matching the video feature according to the video feature, the video generation material including video material or picture material;

[0148] S103: acquiring content text corresponding to the video generation material based on the video feature, wherein the content text is used to generate a video clip together with the corresponding video generation material.

[0149] In this embodiment, obtaining the video theme is the initial step of video generation, and the video theme determines the overall direction and content of the video. The theme can be input by the user, automatically generated by the system, or selected from a preset theme library. The video theme is usually based on the application scenario, target audience or specific needs of the video (such as product promotion, market analysis, etc.).

[0150] The system can obtain video topics in a variety of ways, such as users manually entering "quarterly market analysis report" as the topic on the platform, or the system automatically generates topics suitable for current business needs through historical data analysis. In the financial field, possible topics include "stock market performance" or "investment strategy recommendation."

[0151] Video features are specific content types, emotional expressions, or scene requirements extracted from the video theme. Content types may include data display, product promotion, education and training, etc. Emotional expressions include motivation, professionalism, and relaxation, etc. Scene requirements determine the specific materials or styles required for the video.

[0152] The system will perform semantic analysis on the input video topic to determine its core features. For example, in a financial video, the topic of "market analysis" may be related to "data presentation" and "professionalism". The system will identify this feature and use it for subsequent material selection and text generation. By analyzing these features, the system can determine that the video needs to highlight certain specific information, such as market data or risk assessment.

[0153] Video-generated materials are the basic elements that constitute video content, and may include video clips or picture materials. The selection of materials should be highly matched with the characteristics of the video to ensure that the video content is consistent with the theme. Video-generated materials may be obtained from a local library, an online resource library, or a customized material library.

[0154] The system matches the material based on the video features and retrieves the video clips or pictures that best match the features. For example, for the topic of "market analysis", the system may retrieve pictures or video clips including stock market trend charts and market fluctuation data. In the financial field, the system can automatically obtain video clips related to market trends from the financial database according to the specific needs of the user.

[0155] The content text corresponds to the video generated material and is used to explain or interpret the content in the video material. The content text can be automatically generated by a large language model or input by the user. The text should be consistent with the theme and characteristics of the video to ensure that the information conveyed is accurate and professional.

[0156] Based on video features, the system uses a large language model to generate content text that matches the video material. For example, the system generates a commentary for a "market analysis report" video, which may include a summary of quarterly performance, risk warnings, and investment advice. Content text in the financial field usually involves market data, product introductions, or risk assessments, and the generated text must accurately reflect these contents.

[0157] Example description: Assume that a financial institution needs to produce a "quarterly market analysis report" video. The system first analyzes the video features of "data display" and "professional explanation" based on the input "market analysis" theme. The system retrieves video clips and pictures containing "quarterly stock performance" and "market risk" from the material library. Then, the system uses a large language model to generate content text to explain the changes and risk warnings of each market data. Finally, these materials and texts are combined to generate a complete market analysis video for customer presentations or internal reporting.

[0158] This embodiment analyzes the video theme and features, automatically matches and generates video materials and content text, reduces the time of manual material selection and script writing, and improves the efficiency of video generation. It ensures that the video content is highly consistent with the theme, especially in the financial field, and can quickly generate video content related to market analysis and product promotion, helping financial institutions improve content generation efficiency and accuracy.

[0159] In one embodiment, the above S30 includes:

[0160] S301, performing time axis analysis on the speech material to extract the start time, end time and speech speed information of each speech segment in the speech material;

[0161] S302, generating corresponding subtitle text based on the start time and end time of each speech segment in the speech material and the corresponding content text;

[0162] S303, adjusting the timeline of the corresponding subtitle text according to the speech speed information of each speech segment;

[0163] S304, configuring the font, size, color and display position of the subtitle text to generate synchronized subtitles with the voice material.

[0164] In this embodiment, the timeline analysis is to process the voice material and extract the start time, end time and speech speed information of each voice segment. These time points ensure that the display duration of the subtitles is consistent with the voice content, and the speech speed information is used to adjust the display speed of the subtitles so that the subtitles can be naturally synchronized with the voice.

[0165] The system automatically analyzes the voice material through audio processing technology, identifies the start and end time of the voice, and extracts the speech speed information. These data provide the basis for generating synchronized subtitles. In the financial field, for example, in the "Market Data Analysis" video, the system extracts the start and end time of each commentary to ensure that the subtitles appear in the correct time period and adjusts the display time of the subtitles according to the speech speed.

[0166] The generation of subtitle text is based on the timeline information of the voice material and the content of the content text. By ensuring that the start and end times of the subtitle text and the voice material are consistent, the system can match each voice commentary content to the corresponding subtitle, achieving synchronous display of voice and subtitles.

[0167] The system segments the content text according to the extracted timeline information and generates corresponding subtitle text for each speech segment. In the financial field, for example, when explaining "market fluctuations", the subtitles will display the corresponding text description, such as "the current market volatility is 5%", and match the playback time of the speech, ensuring that the audience can see the subtitles and hear the explanation at the same time.

[0168] Speech speed directly affects the display duration of subtitles. Based on the speech speed information, the system adjusts the timeline of the subtitle text to ensure that the display speed of the subtitles adapts to the playback speed of the speech, avoiding the subtitles being displayed too fast or too slow, which affects the audience's understanding.

[0169] The system automatically adjusts the subtitle display timeline to match the speech speed of the voice material. For example, in the "Financial Market Report" video, if the commentary is fast, the system will shorten the subtitle display time to ensure that the subtitles and voice are played synchronously; conversely, if the speech speed is slow, the system will extend the subtitle display time.

[0170] The configuration of subtitles includes the settings of font style, size, color and display position. Subtitles must be readable in the video screen and not interfere with the visual effect of the video content. At the same time, the subtitle style must be consistent with the overall visual style of the video.

[0171] The system will automatically configure the subtitle style based on the video content, such as using a more stable font, appropriate color and size to ensure that the subtitles are clearly visible on the screen. For example, in a financial video, the system can choose a simple white font and place it at the bottom of the screen, making the subtitles clear and readable without interfering with the data display.

[0172] This embodiment greatly improves the efficiency of video production by automatically generating subtitles that are synchronized with the voice material, avoiding the complicated process of manually adding and adjusting subtitles. The system can accurately adjust the timeline and style of subtitles to ensure that subtitles are synchronized with the voice, so that the audience can better understand the video content.

[0173] In one embodiment, an automatic video generation device is provided, and the automatic video generation device corresponds one-to-one to the automatic video generation method in the above embodiment. Figure 3 , Figure 3This is a functional module diagram of a preferred embodiment of the automatic video generation device of the present invention. Video generation material acquisition module 10, voice material generation module 20, subtitle generation and synchronization module 30, video segment generation module 40 and video synthesis module 50. The functional modules are described in detail as follows:

[0174] The video generation material acquisition module 10 is used to determine the video theme, and acquire a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials;

[0175] A voice material generation module 20, used to convert each content text into voice data through a text-to-speech model, and generate corresponding voice materials, which are used to generate video clips;

[0176] A subtitle generation and synchronization module 30, for generating corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material;

[0177] A video segment generation module 40, for generating independent video segments based on each video generation material and corresponding voice material and synchronized subtitles;

[0178] The video synthesis module 50 is used to merge all independent video clips into a complete video to obtain a target video.

[0179] In one embodiment, the video segment generation module 40 is specifically configured to:

[0180] If the video generation material is a video material, a large language model is used to summarize the content text corresponding to the video generation material to form key information;

[0181] Based on the key information, use video moment retrieval technology to find and extract key video clips in the video generation material;

[0182] Processing the extracted key video clips to remove the original sound in the key video clips;

[0183] Synchronously embedding the voice material and the synchronized subtitles into the processed key video clip;

[0184] The processed key video clips are rendered using video processing tools to generate independent video clips including video materials, voice materials and synchronized subtitles.

[0185] In one embodiment, the video segment generation module 40 is specifically configured to:

[0186] If the video generation material is a picture material, a large language model is used to summarize the content text corresponding to the video generation material to form key information;

[0187] Using a picture-text matching model to search the video generation material and determine a key picture matching the key information;

[0188] The key images are converted into continuous dynamic video segments through transition effects;

[0189] Synchronously embedding the voice material and the synchronized subtitles into the dynamic video segment;

[0190] The dynamic video segment is rendered to generate an independent video segment including picture materials, voice materials and synchronized subtitles.

[0191] In one embodiment, the video segment generation module 40 is specifically configured to:

[0192] Matching video background music according to the video theme;

[0193] Determine the volume of the video background music according to the clarity of the voice material and the video scene requirements;

[0194] Creating an audio track, and synchronously embedding the video background music and the voice material into the audio track;

[0195] Render the video clip to generate an independent video clip containing video background music, video generation material, corresponding voice material and synchronized subtitles.

[0196] In one embodiment, the video synthesis module 50 is specifically used for:

[0197] Sort all generated independent video clips according to video theme, timeline or scene order;

[0198] Use video editing tools to splice each independent video clip and process the transition effects between independent video clips;

[0199] Merge the audio tracks of each independent video clip and handle the connection between voice materials;

[0200] Render the spliced ​​video and merged audio tracks to generate the complete target video.

[0201] In one embodiment, the video generation material acquisition module 10 is specifically used for:

[0202] Acquire a video theme, analyze the video theme to determine video features, the video features including content type, emotional expression or scene requirements;

[0203] According to the video feature, obtaining a video generation material matching the video feature, the video generation material including a video material or a picture material;

[0204] A content text corresponding to the video generation material is acquired based on the video feature, and the content text is used to generate a video clip together with the corresponding video generation material.

[0205] In one embodiment, the subtitle generation and synchronization module 30 is specifically used for:

[0206] Perform timeline analysis on the speech material to extract the start time, end time and speech speed information of each speech segment in the speech material;

[0207] Generate corresponding subtitle text based on the start and end time of each speech segment in the speech material and the corresponding content text;

[0208] Adjust the timeline of the corresponding subtitle text according to the speech speed information of each voice segment;

[0209] Configure the font, size, color, and display position of the subtitle text to generate synchronized subtitles with the voice material.

[0210] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal via a network connection. When the computer program is executed by the processor, the functions or steps on the server side of an automated video generation method are implemented.

[0211] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a user side of an automated video generation method

[0212] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0213] Determine a video theme, and obtain a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials;

[0214] Convert each content text into speech data through a text-to-speech model to generate corresponding speech material, which is used to generate video clips;

[0215] Generate corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material;

[0216] Generate independent video clips based on each video generation material and corresponding voice material and synchronized subtitles;

[0217] Merge all independent video clips into a complete video to obtain the target video.

[0218] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0219] Determine a video theme, and obtain a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials;

[0220] Convert each content text into speech data through a text-to-speech model to generate corresponding speech material, which is used to generate video clips;

[0221] Generate corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material;

[0222] Generate independent video clips based on each video generation material and corresponding voice material and synchronized subtitles;

[0223] Merge all independent video clips into a complete video to obtain the target video.

[0224] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0225] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0226] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0227] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An automated video generation method, characterized in that: The following steps are involved: Determine a video theme, and obtain a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials; Convert each content text into speech data through a text-to-speech model to generate corresponding speech material, which is used to generate video clips; Generate corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material; Generate independent video clips based on each video generation material and corresponding voice material and synchronized subtitles; Merge all independent video clips into a complete video to obtain the target video.

2. The automatic video generation method according to claim 1, characterized in that: Generate independent video clips based on each video generation material and the corresponding voice material and synchronized subtitles, including: If the video generation material is a video material, a large language model is used to summarize the content text corresponding to the video generation material to form key information; Based on the key information, use video moment retrieval technology to find and extract key video clips in the video generation material; Processing the extracted key video clips to remove the original sound in the key video clips; Synchronously embedding the voice material and the synchronized subtitles into the processed key video clip; The processed key video clips are rendered using video processing tools to generate independent video clips including video materials, voice materials and synchronized subtitles.

3. The automatic video generation method according to claim 1, characterized in that: Generate independent video clips based on each video generation material and the corresponding voice material and synchronized subtitles, including: If the video generation material is a picture material, a large language model is used to summarize the content text corresponding to the video generation material to form key information; Using a picture-text matching model to search the video generation material and determine a key picture matching the key information; The key images are converted into continuous dynamic video segments through transition effects; Synchronously embedding the voice material and the synchronized subtitles into the dynamic video segment; The dynamic video segment is rendered to generate an independent video segment including picture materials, voice materials and synchronized subtitles.

4. The automatic video generation method according to claim 1, characterized in that: Generate independent video clips based on each video generation material and the corresponding voice material and synchronized subtitles, including: Matching video background music according to the video theme; Determine the volume of the video background music according to the clarity of the voice material and the video scene requirements; Creating an audio track, and synchronously embedding the video background music and the voice material into the audio track; Render the video clip to generate an independent video clip containing video background music, video generation material, corresponding voice material and synchronized subtitles.

5. The automatic video generation method according to claim 1, characterized in that: Merge all independent video clips into a complete video to get the target video, including: Sort all generated independent video clips according to video theme, timeline or scene order; Use video editing tools to splice each independent video clip and process the transition effects between independent video clips; Merge the audio tracks of each independent video clip and handle the connection between voice materials; Render the spliced ​​video and merged audio tracks to generate the complete target video.

6. The automatic video generation method as claimed in claim 1, characterized in that: Determine a video theme, and obtain a plurality of video generation materials and a plurality of content texts corresponding to the video theme, including: Acquire a video theme, analyze the video theme to determine video features, the video features including content type, emotional expression or scene requirements; According to the video feature, obtaining a video generation material matching the video feature, the video generation material including a video material or a picture material; A content text corresponding to the video generation material is acquired based on the video feature, and the content text is used to generate a video clip together with the corresponding video generation material.

7. The automatic video generation method according to claim 1, characterized in that: Generating corresponding synchronized subtitles based on each voice material, wherein the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material, including: Perform timeline analysis on the speech material to extract the start time, end time and speech speed information of each speech segment in the speech material; Generate corresponding subtitle text based on the start and end time of each speech segment in the speech material and the corresponding content text; Adjust the timeline of the corresponding subtitle text according to the speech speed information of each voice segment; Configure the font, size, color, and display position of the subtitle text to generate synchronized subtitles with the voice material.

8. An automatic video generation device, characterized in that: The automatic video generation device comprises: A video generation material acquisition module, used to determine a video theme, and acquire a plurality of video generation materials and a plurality of content texts corresponding to the video theme, wherein the video generation materials include video materials or picture materials, and the content texts are used to generate video clips together with the corresponding video generation materials; A voice material generation module, used to convert each content text into voice data through a text-to-speech model, and generate corresponding voice materials, which are used to generate video clips; A subtitle generation and synchronization module, used to generate corresponding synchronized subtitles based on each voice material, and the synchronized subtitles are used to be embedded in the video clip synchronously with the voice material; A video segment generation module, used to generate independent video segments based on each video generation material and corresponding voice material and synchronized subtitles; The video synthesis module is used to merge all independent video clips into a complete video to obtain the target video.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and an automated video generation program stored in the memory and executable on the processor. When the automated video generation program is executed by the processor, the steps of the automated video generation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores an automated video generation program, which, when executed by a processor, implements the steps of the automated video generation method according to any one of claims 1 to 7.

Images