Method and device for converting text into video, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a method and device for converting a text into a video, electronic equipment and a storage medium, and the method comprises the steps: obtaining text data, and segmenting the text data into a plurality of text segments which accord with a preset segment length; utilizing a customized Prompt template to respectively convert the plurality of text segments into drawing instruction text data in a format which can be identified by an AI drawing tool; calling a text-to-voice conversion API (Application Program Interface), and respectively converting the plurality of text segments into voice data matched with the text content; calling an AI drawing service, and generating a picture sequence synchronized with the voice data according to the drawing instruction text data; and extracting duration information from voice data corresponding to the text data, setting the display duration of each picture in the picture sequence in the video based on the duration information, and generating a video file based on the voice data and the picture sequence by adopting a video editing tool.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for converting text into video. Background Art

[0002] With the rapid development of digital media and artificial intelligence technology, automated video generation technology is gradually becoming a hot spot for innovation. In the existing technology, traditional video production is very cumbersome and costly, so there is an urgent need for an efficient and automated video generation solution to meet the market demand for fast multimedia content production. Summary of the invention

[0003] The object of the present invention is to provide a method, device, electronic device and storage medium for converting text into video, aiming to solve the above-mentioned problems in the prior art.

[0004] The present invention provides a method for converting text into video, comprising:

[0005] Acquire text data, and divide the text data into a plurality of text segments having preset segment lengths;

[0006] Using a customized Prompt template, the multiple text fragments are respectively converted into drawing instruction text data in a format recognizable by the AI ​​drawing tool;

[0007] Calling a text-to-speech conversion API to convert the multiple text segments into speech data that matches the text content;

[0008] Calling an AI drawing service to generate a picture sequence synchronized with the voice data according to the drawing instruction text data;

[0009] Duration information is extracted from the voice data corresponding to the text data, and based on the duration information, the display duration of each picture in the picture sequence in the video is set, and a video file is generated based on the voice data and the picture sequence using a video editing tool.

[0010] The present invention provides a device for converting text into video, comprising:

[0011] A segmentation module, used to obtain text data and segment the text data into a plurality of text segments having preset segment lengths;

[0012] A first conversion module, configured to use a customized Prompt template to convert the plurality of text segments into drawing instruction text data in a format recognizable by an AI drawing tool;

[0013] A second conversion module, configured to call a text-to-speech conversion API to convert the plurality of text segments into speech data that matches the text content;

[0014] A picture generation module, used to call the AI ​​drawing service and generate a picture sequence synchronized with the voice data according to the drawing instruction text data;

[0015] A video generation module is used to extract duration information from the voice data corresponding to the text data, set the display duration of each picture in the picture sequence in the video based on the duration information, and use a video editing tool to generate a video file based on the voice data and the picture sequence.

[0016] An embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned method of converting text into video when executed by the processor.

[0017] An embodiment of the present invention further provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by a processor, the steps of the above-mentioned method for converting text into video are implemented.

[0018] By adopting the embodiments of the present invention, text information is converted into a video format that is both visually and auditorily pleasing, so that automatic conversion of text content into video content can be achieved and the continuity and high-quality experience of the video content can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 is a flow chart of a method for converting text into video according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of an apparatus for converting text into video according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0024] Method Embodiment

[0025] According to an embodiment of the present invention, a method for converting text into video is provided. Figure 1 is a flowchart of a method for converting text into video according to an embodiment of the present invention. Figure 1 As shown, the method for converting text into video according to an embodiment of the present invention specifically includes:

[0026] Step S101, obtaining text data, and dividing the text data into a plurality of text segments that meet a preset segment length; in an embodiment of the present invention, dividing the text data into a plurality of text segments that meet the preset segment length specifically includes:

[0027] Using line breaks, periods, exclamation marks, semicolons, tildes, and question marks as delimiters, the text data is divided into preliminary text segments;

[0028] The length of the preliminary text fragment is evaluated. If the fragment length is less than or equal to the preset fragment length, the preliminary text fragment is saved. If the length exceeds the preset fragment length, the preliminary text fragment is divided into two stages using commas, colons, double quotes, equal signs, and single quotes as delimiters; preferably, the preset fragment length is 20 characters.

[0029] For text segments that still exceed the preset segment length after the second stage segmentation, use the pound sign, & symbol, asterisk, slash and backslash as separators to perform three-stage segmentation until multiple text segments suitable for speech synthesis and video presentation are obtained.

[0030] Step S102, using a customized prompt template, converting the multiple text fragments into drawing instruction text data in a format recognizable by the AI ​​drawing tool; the customized prompt template is continuously adjusted according to the business situation

[0031] Step S103, calling the text-to-speech conversion API to convert the multiple text segments into voice data that matches the text content; specifically, the composite technology that can be used for the above processing includes the open source repository edge_tts, and EmotiVoice, etc. If there is a tool with better effect, it can also be used. It mainly processes text-to-speech, and uses the ability of voice to carry emotions.

[0032] Step S104, calling an AI drawing service to generate a picture sequence synchronized with the voice data according to the drawing instruction text data;

[0033] Step S105, extracting duration information from the voice data corresponding to the text data, setting the display duration of each picture in the picture sequence in the video based on the duration information, and using a video editing tool to generate a video file based on the voice data and the picture sequence.

[0034] In an embodiment of the present invention, using a video editing tool to generate a video file based on the voice data and the picture sequence specifically includes:

[0035] Using the voice data as the audio track of the video;

[0036] The image sequence is added to the video timeline according to the set display duration, a smooth transition effect is applied to the image sequence, and color correction and brightness adjustment are performed to generate a video file.

[0037] In summary, with the help of the technical solution of the embodiment of the present invention, this technology greatly reduces the need for manual editing and significantly improves the efficiency of video production. The text disassembly algorithm and customized prompt translation mechanism of the embodiment of the present invention provide a new solution for the field of video generation. Through precise voice and picture synchronization technology, the embodiment of the present invention ensures the smooth display of video content. The technical solution of the embodiment of the present invention meets the market demand for fast, efficient and low-cost video production solutions. By combining advanced text processing technology and AI drawing capabilities, the embodiment of the present invention has broad application potential in multiple industries, indicating huge market prospects.

[0038] The embodiments of the present invention have a wide range of application scenarios, for example, 1. Children's educational content: to make children's educational videos, AI can generate images with rich colors and simple shapes, which are suitable for children's cognitive level. 2. Marketing promotion: to quickly make marketing videos to show product features in a dynamic and attractive way. 3. Entertainment media: to automatically convert literary works or scripts into video content for entertainment and media publishing. 4. Fictional stories and novels: to generate videos for novels, comics or fictional stories, AI can create attractive visual elements. 5. Emotional and inspirational content: to generate inspirational speeches, emotional stories or soul-stirring videos.

[0039] The above technical solution of the embodiment of the present invention is described in detail below.

[0040] The present invention proposes a text segmentation based on complex rules, which uses punctuation marks and string length restrictions to decompose long text into short segments suitable for video scripts. The Prompt design mechanism is used to convert the disassembled text into a format that can be parsed by AI drawing tools. Through the collaborative mechanism of speech synthesis and image generation, the fluency of video content is ensured through precise timestamp matching. Professional tools such as FFMpeg are used to intelligently set the image display time according to the speech duration to synthesize professional quality videos.

[0041] Step 1: Acquire text data.

[0042] Input: text entered by the user or text data read from a local file.

[0043] Output: Raw, unprocessed text data.

[0044] Step 2: Decompose the text content.

[0045] Input: Raw text data.

[0046] The first stage of decomposition: Use line breaks, periods, exclamation marks, semicolons, tildes, question marks, etc. as the main separators, and divide the text into a preliminary list of fragments based on these symbols.

[0047] Length assessment: The length of the text fragment obtained by the first stage of decomposition is assessed. If the length of the fragment is less than or equal to 20 characters, it is considered that the fragment is suitable for subsequent processing; if the length exceeds 20 characters, it enters the second stage of decomposition.

[0048] Second stage of decomposition: For text segments longer than 20 characters, further decomposition is performed using second-order symbols such as commas, colons, double quotes, equal signs, and single quotes until all segments are of moderate length.

[0049] The third stage of decomposition: For text segments that still have more than 20 characters after the second stage of decomposition, third-order symbols such as pound sign, & symbol, asterisk, slash, backslash, etc. are used to decompose them until a text segment suitable for speech synthesis and video display is obtained.

[0050] Output: A series of text snippets of appropriate length, suitable for speech synthesis and video presentation.

[0051] Step 3, prompt translation.

[0052] Input: List of optimized text snippets.

[0053] Method: Use a customized Prompt template to convert text snippets into a format that the AI ​​drawing tool can recognize.

[0054] Output: Text data converted into drawing instructions.

[0055] Step 4, text to speech conversion.

[0056] Input: List of optimized text snippets.

[0057] Method: Call the text-to-speech conversion API to convert text into speech data.

[0058] Output: Voice data that matches the text content.

[0059] Step 5: Image generation.

[0060] Input: Drawing instruction text data.

[0061] Method: Call AI drawing service and generate pictures according to drawing instructions.

[0062] Output: A sequence of pictures synchronized with the speech data.

[0063] Step 6: Video synthesis.

[0064] Input: This step receives the speech data and picture sequence generated by the previous step.

[0065] Method: Timeline synchronization: First, extract the duration information by reading the speech file corresponding to the text.

[0066] Image duration setting: Use the length of the extracted voice file to set the display duration of each image in the video to ensure that the image display is synchronized with the voice playback.

[0067] Video editing tool application: Use professional video editing tools such as FFMpeg to perform the following operations:

[0068] 1. Use the voice data as the audio track of the video.

[0069] 2. Add the image sequence to the video timeline according to the set duration.

[0070] 3. Apply smooth transition effects to enhance visual continuity.

[0071] 4. Perform color correction and brightness adjustment to ensure the consistency of the picture with the overall video style.

[0072] 5. Implement final video encoding and format conversion to generate high-quality video files suitable for different playback platforms.

[0073] Output: The output is a carefully synchronized and edited video file with professional quality. The video accurately matches the voice and visual content, provides a smooth viewing experience, and is compatible with mainstream video playback standards.

[0074] Through the above blowing, accurate synchronization can be achieved: through precise timeline control, the synchronization of voice and picture can be ensured to avoid the problem of asynchrony between audio and picture. High-quality output can be achieved: using the advanced editing functions of tools such as FFMpeg, the output video has professional-level visual effects and sound quality.

[0075] Device Example 1

[0076] According to an embodiment of the present invention, a device for converting text into video is provided. Figure 2 is a schematic diagram of an apparatus for converting text into video according to an embodiment of the present invention, such as Figure 2 As shown, the device for converting text into video according to an embodiment of the present invention specifically includes:

[0077] The segmentation module 20 is used to obtain text data and segment the text data into a plurality of text segments that meet preset segment lengths; specifically, to:

[0078] Using line breaks, periods, exclamation marks, semicolons, tildes, and question marks as delimiters, the text data is divided into preliminary text segments;

[0079] The length of the preliminary text fragment is evaluated. If the fragment length is less than or equal to the preset fragment length, the preliminary text fragment is saved. If the length exceeds the preset fragment length, the preliminary text fragment is divided into two stages using commas, colons, double quotes, equal signs, and single quotes as delimiters; the preset fragment length is 20 characters.

[0080] For text segments that still exceed the preset segment length after the second stage segmentation, use the pound sign, & symbol, asterisk, slash and backslash as separators to perform three-stage segmentation until multiple text segments suitable for speech synthesis and video presentation are obtained.

[0081] A first conversion module 22, used to convert the plurality of text segments into drawing instruction text data in a format recognizable by an AI drawing tool, using a customized Prompt template;

[0082] A second conversion module 24 is used to call a text-to-speech conversion API to convert the multiple text segments into speech data that matches the text content;

[0083] The picture generation module 26 is used to call the AI ​​drawing service and generate a picture sequence synchronized with the voice data according to the drawing instruction text data;

[0084] The video generation module 28 is used to extract duration information from the voice data corresponding to the text data, set the display duration of each picture in the picture sequence in the video based on the duration information, and use a video editing tool to generate a video file based on the voice data and the picture sequence. The video generation module 28 is specifically used to:

[0085] Using the voice data as the audio track of the video;

[0086] The image sequence is added to the video timeline according to the set display duration, a smooth transition effect is applied to the image sequence, and color correction and brightness adjustment are performed to generate a video file.

[0087] The embodiment of the present invention is a device embodiment corresponding to the above method embodiment. The specific operations of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.

[0088] Device Example 2

[0089] An embodiment of the present invention provides an electronic device, such as Figure 3 As shown, it includes: a memory 30, a processor 32, and a computer program stored in the memory 30 and executable on the processor 32. When the computer program is executed by the processor 32, the steps described in the method embodiment are implemented.

[0090] Device Example 3

[0091] An embodiment of the present invention provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by the processor 32, the steps described in the method embodiment are implemented.

[0092] The computer-readable storage medium in this embodiment includes, but is not limited to, ROM, RAM, magnetic disk or optical disk, etc.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for converting text into video, characterized in that: include: Acquire text data, and divide the text data into a plurality of text segments having preset segment lengths; Using a customized Prompt template, the multiple text fragments are respectively converted into drawing instruction text data in a format recognizable by the AI ​​drawing tool; Calling a text-to-speech conversion API to convert the multiple text segments into speech data that matches the text content; Calling an AI drawing service to generate a picture sequence synchronized with the voice data according to the drawing instruction text data; Duration information is extracted from the voice data corresponding to the text data, and based on the duration information, the display duration of each picture in the picture sequence in the video is set, and a video file is generated based on the voice data and the picture sequence using a video editing tool.

2. The method according to claim 1, characterized in that Segmenting the text data into a plurality of text segments that meet preset segment lengths specifically includes: Using line breaks, periods, exclamation marks, semicolons, tildes, and question marks as delimiters, the text data is segmented into preliminary text segments; The length of the preliminary text segment is evaluated. If the segment length is less than or equal to a preset segment length, the preliminary text segment is saved. If the length exceeds the preset segment length, a comma, a colon, a double quotation mark, an equal sign, and a single quotation mark are used as delimiters to perform a two-stage segmentation on the preliminary text segment. For text segments that still exceed the preset segment length after the second stage segmentation, use the pound sign, & symbol, asterisk, slash and backslash as separators to perform three-stage segmentation until multiple text segments suitable for speech synthesis and video presentation are obtained.

3. The method according to claim 2, characterized in that The preset segment length is a character length determined according to actual conditions.

4. The method according to claim 1, characterized in that: Using a video editing tool to generate a video file based on the voice data and the picture sequence specifically includes: Using the voice data as the audio track of the video; The image sequence is added to the video timeline according to the set display duration, a smooth transition effect is applied to the image sequence, and color correction and brightness adjustment are performed to generate a video file.

5. A device for converting text into video, characterized in that: include: A segmentation module, used to obtain text data and segment the text data into a plurality of text segments having preset segment lengths; A first conversion module, configured to use a customized Prompt template to convert the plurality of text segments into drawing instruction text data in a format recognizable by an AI drawing tool; A second conversion module, configured to call a text-to-speech conversion API to convert the plurality of text segments into speech data that matches the text content; A picture generation module, used to call the AI ​​drawing service and generate a picture sequence synchronized with the voice data according to the drawing instruction text data; A video generation module is used to extract duration information from the voice data corresponding to the text data, set the display duration of each picture in the picture sequence in the video based on the duration information, and use a video editing tool to generate a video file based on the voice data and the picture sequence.

6. The device according to claim 5, characterized in that The segmentation module is specifically used for: Using line breaks, periods, exclamation marks, semicolons, tildes, and question marks as delimiters, the text data is segmented into preliminary text segments; The length of the preliminary text segment is evaluated. If the segment length is less than or equal to a preset segment length, the preliminary text segment is saved. If the length exceeds the preset segment length, a comma, a colon, a double quotation mark, an equal sign, and a single quotation mark are used as delimiters to perform a two-stage segmentation on the preliminary text segment. For text segments that still exceed the preset segment length after the second stage segmentation, use the pound sign, & symbol, asterisk, slash and backslash as separators to perform three-stage segmentation until multiple text segments suitable for speech synthesis and video presentation are obtained.

7. The device according to claim 6, characterized in that The preset segment length is a character length determined according to actual conditions.

8. The device according to claim 5, characterized in that The video generation module is specifically used for: Using the voice data as the audio track of the video; The image sequence is added to the video timeline according to the set display duration, a smooth transition effect is applied to the image sequence, and color correction and brightness adjustment are performed to generate a video file.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for converting text into video according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the method for converting text into video according to any one of claims 1 to 4 are implemented.

Images