Method and device for generating video from audio, equipment and storage medium

Abstract

The invention relates to the technical field of audio and video data processing, and provides a method and device for generating a video from an audio, equipment and a storage medium, and the method comprises the steps: segmenting a to-be-processed audio file, and obtaining at least two sub-audio files; for any sub-audio file, converting the sub-audio file into at least two pieces of text data; key nouns of the text data are extracted, and first images of the key nouns are determined according to the key nouns; according to the first image of each key noun, the sub-audio file and the corresponding text data, generating a first sub-audio / video file corresponding to the sub-audio file; and combining the first sub-audio and video files corresponding to the sub-audio files to obtain a target video. According to the invention, the key noun is extracted based on the text data converted from the sub-audio file, and then the first image is generated based on the key noun, so that the image generation process is simple and convenient, the video generation efficiency is improved, and the audio and video display in the generated target video is synchronous.

Description

Technical Field

[0001] The present invention relates to the technical field of audio and video data processing, and in particular to a method, device, equipment and storage medium for generating video from audio. Background Art

[0002] An audio file with only voice information is far less attractive to viewers than an audio file with video content. In order to improve the viewing experience of an audio file, it is often necessary to equip the audio file with video content.

[0003] In the prior art, most of the existing video production methods require manual shooting of video materials or searching for corresponding video materials, overall identification of the audio files to be processed, and then matching the audio files to be processed with the manually shot or searched video materials to generate a video corresponding to the audio files to be processed. The efficiency of video generation is low and the audio and video display in the generated video is not synchronized. Summary of the invention

[0004] In view of the above-mentioned defects in the prior art, the present invention provides a method, device, equipment and storage medium for generating video from audio, which improves the efficiency of video generation and ensures that the audio and video display in the generated target video is synchronized.

[0005] In a first aspect, the present invention provides a method for generating video from audio, the method comprising the following steps: Splitting the audio file to be processed to obtain at least two sub-audio files; For any of the sub-audio files, convert the sub-audio file into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; Generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

[0006] According to a method for generating video from audio provided by the present invention, the step of extracting key terms from each of the text data comprises: Performing word segmentation on each of the text data to obtain a word segmentation result corresponding to each of the file data; the word segmentation result includes at least one word; Performing part-of-speech tagging on each of the words to obtain part-of-speech tagging results for each of the words; Extracting nouns from the text data according to the part-of-speech tagging results of the words; The nouns in each of the text data are prioritized to obtain key nouns in each of the text data; the priority of the noun is used to represent the priority of the matching degree of the noun representing the text data.

[0007] According to a method for generating video from audio provided by the present invention, the step of determining the first image of each key noun according to each key noun comprises: According to each of the key nouns, a first prompt word for generating an image is determined; the first prompt word is an input instruction of a large image model; the large image model is obtained by pre-training the image model using a labeled data set; the labeled data set includes a plurality of second prompt words for generating an image and labeled images generated based on each of the second prompt words; According to the first prompt word, the first image of each of the key nouns is generated using the image macromodel.

[0008] According to a method for generating video from audio provided by the present invention, the step of determining the first image of each key noun according to each key noun comprises: Matching each of the key nouns in a pre-acquired video file library to obtain at least two second images matching each of the key nouns; According to each of the second images and a preset rule, a first image of each of the key nouns is determined.

[0009] According to a method for generating video from audio provided by the present invention, determining the first image of each key noun according to each second image and a preset rule includes: Screening each of the second images according to the preset rule to obtain each of the third images that meets the preset rule; Sorting the resolution of each of the third images to obtain a resolution sorting result of each of the third images; The first image of each of the key nouns is obtained according to the resolution sorting result of each of the third images.

[0010] According to a method for generating video from audio provided by the present invention, the step of segmenting an audio file to be processed to obtain at least two sub-audio files includes: Determine at least one speech pause point of the audio file to be processed; Determine each of the speech pause points as each segmentation point of the audio file to be processed; Based on each of the segmentation points, the audio file to be processed is segmented into sub-audio files corresponding to each of the segmentation points; The sub-audio files corresponding to the segmentation points are determined as the at least two sub-audio files.

[0011] According to a method for generating video from audio provided by the present invention, generating a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data, comprises: synthesizing the first image of each of the key nouns with each of the sub-audio files to obtain a second sub-audio video file corresponding to each of the sub-audio files; According to each of the second sub-audio and video files and each of the text data corresponding to each of the sub-audio files, a first sub-audio and video file corresponding to each of the sub-audio files is obtained.

[0012] According to a method for generating video from audio provided by the present invention, obtaining a first sub-audio and video file corresponding to each sub-audio file according to each of the second sub-audio and video files and each of the text data corresponding to each of the sub-audio files includes: Encoding each of the second sub-audio and video files to obtain encoded each of the second sub-audio and video files; According to the encoded second sub-audio and video files and the text data corresponding to the sub-audio files, the first sub-audio and video files corresponding to the sub-audio files are obtained.

[0013] According to a method for generating video from audio provided by the present invention, encoding each of the second sub-audio and video files to obtain each encoded second sub-audio and video file includes: Get the video encoding format, key frame interval and audio encoding format; According to the video encoding format, the key frame interval and the audio encoding format, the second sub-audio and video files are sequentially encoded to obtain the encoded second sub-audio and video files.

[0014] According to a method for generating video from audio provided by the present invention, merging the first sub-audio and video files corresponding to the sub-audio files to obtain a target video includes: In the case where there are multiple first sub-audio and video files, obtaining the encoding order of the multiple first sub-audio and video files; the encoding order of the multiple first sub-audio and video files is consistent with the encoding order of the encoded second sub-audio and video files; The plurality of the first sub-audio and video files are merged according to the encoding order to obtain the target video.

[0015] In a second aspect, the present invention further provides a device for generating video from audio, the device comprising the following modules: An audio segmentation module, used for segmenting the audio file to be processed to obtain at least two sub-audio files; An image generation module, used for converting any of the sub-audio files into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; A video generation module, used for generating a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

[0016] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method of generating video from audio as described above is implemented.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for generating video from audio.

[0018] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for generating video from audio.

[0019] The method, device, equipment and storage medium for generating video from audio provided by the present invention divide the audio file to be processed to obtain at least two sub-audio files, and then, for any sub-audio file, convert the sub-audio file into at least two text data, extract the key nouns of each text data, and determine the first image of each key noun according to each key noun; then, according to the first image of each key noun, the sub-audio file and the corresponding text data, generate the first sub-audio and video files corresponding to the sub-audio file; merge the first sub-audio and video files corresponding to each sub-audio file to obtain the target video.

[0020] In the present invention, the audio files to be processed that need to be equipped with videos are first segmented, and then speech-to-text conversion and key noun extraction are performed on each sub-audio file, and a corresponding first image is generated according to the key noun. Further, a target video is generated according to the first image of each key noun corresponding to each sub-audio file, each sub-audio file, and each text data corresponding to each sub-audio file. In the present invention, key nouns are first extracted based on the text data converted from the sub-audio files, and then the first image is generated based on the key noun. The process of generating an image is simple, thereby improving the efficiency of video generation and achieving synchronized audio and video display in the generated target video. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 This is one of the flow charts of the method for generating video from audio provided by the present invention.

[0023] Figure 2 This is the second flow chart of the method for generating video from audio provided by the present invention.

[0024] Figure 3 It is a structural schematic diagram of the video generation system provided by the present invention.

[0025] Figure 4 It is a structural schematic diagram of the video generating device provided by the present invention.

[0026] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally a class, and the number of objects is not limited. For example, the first node can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the front and back associated objects are in an "or" relationship.

[0029] Combine the following Figure 1-Figure 5 The present invention describes the method, device, apparatus and storage medium for generating video from audio.

[0030] Figure 1FIG. 1 is one of the flow charts of the method for generating video from audio provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 101, splitting the audio file to be processed to obtain at least two sub-audio files; Specifically, the executor of this embodiment is an electronic device, and this embodiment is used to improve the generation efficiency of speech-generated video.

[0031] First, obtain the audio file to be processed for generating the video, wherein there are usually the following ways to obtain the audio file to be processed: Recording equipment: Use a microphone, voice recorder or other recording equipment to record sound. Computer recording software: Use recording software on the computer, such as Audacity, Adobe Audition, etc., to record. Audio download: Download audio files from the Internet, such as music, podcasts, audio books, etc. Audio library: Obtain audio materials from professional audio libraries or resource websites, which usually provide various types of audio files for download. It should be noted that the method for obtaining the audio file to be processed is determined according to actual needs, and this embodiment does not specifically limit the method for obtaining the audio file to be processed.

[0032] Furthermore, after obtaining the audio file to be processed, since the originally obtained audio file to be processed usually has a large data volume and a long audio duration, in order to better realize the steps of speech-to-text conversion, keyword extraction, and image generation, the audio file to be processed can be pre-segmented to obtain multiple sub-audio files.

[0033] Step 102: for any sub-audio file, convert the sub-audio file into at least two pieces of text data; Specifically, after obtaining multiple sub-audio files, the operation of generating a video from each sub-audio file is similar, and the operation of generating a video from speech can be performed on each sub-audio file one by one. Therefore, the steps of generating a video from speech are introduced below by taking one of the sub-audio files as an example.

[0034] First, the sub-audio file is converted into audio to obtain at least two text data (ie, one or more text records). The process of converting audio into text is usually called audio transcription or speech recognition.

[0035] Exemplarily, the audio conversion process can be implemented by the following methods: 1. Use audio-to-text software: such as audio editor Audacity, audio processing software Adobe Audition; 2. Use open source tools, such as Mozilla DeepSpeech software DeepSpeech, Kaldi, where Mozilla DeepSpeech is an open source speech recognition system that can train its own speech recognition model; 3. Use the Application Programming Interface (API) service: for example, use the speech recognition application interface G Cloud Speech-to-Text API provided by a company, and integrate the speech recognition application interface G Cloud Speech-to-Text API into the electronic device application. It should be noted that which audio conversion method is adopted depends on the specific implementation of the electronic device, and this embodiment does not limit this.

[0036] Step 103, extracting key nouns from each text data, and determining the first image of each key noun according to each key noun; Specifically, for each text data (i.e., text record), key nouns in each text data can be extracted, wherein nouns are word classes representing entities or abstract concepts, and key nouns are used to represent core key entities in the text data, such as the sun, mountain peaks, birds, etc. The method for extracting key nouns is, for example, the keyword extraction method in Jieba.

[0037] Furthermore, the first image corresponding to each text data can be determined based on the key nouns in each text data. For example, the first image corresponding to the key noun is selected from a pre-acquired general video material library, or a conversation is conducted between prompt words and the image model to generate the first image corresponding to each key noun.

[0038] By analogy, the first image corresponding to each text data can be obtained, and then the first image of each key noun corresponding to the sub-audio file can be determined according to the first image corresponding to each text data.

[0039] Step 104: Generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, each sub-audio file and each corresponding text data; Specifically, referring to the above steps 102 to 103, similar operations are performed for each sub-audio file to obtain the first image of each keyword corresponding to each sub-audio file.

[0040] Furthermore, the first sub-audio video file corresponding to the sub-audio file can be generated based on the first image of each key noun corresponding to the sub-audio file, each sub-audio file, and each text data corresponding to the sub-audio file. For example, the first image of each key noun corresponding to each sub-audio file and each sub-audio file are synthesized to obtain a sub-audio video file with the same duration as each sub-audio file. Furthermore, subtitles can be added to each sub-audio video file based on each text data corresponding to each sub-audio file to obtain the first sub-audio video file.

[0041] Step 105: merge the first sub-audio and video files corresponding to each sub-audio file to obtain a target video.

[0042] Specifically, each first sub-audio and video file is merged to obtain a target video, thereby achieving generation of the target video.

[0043] The method provided in this embodiment obtains at least two sub-audio files by segmenting the audio file to be processed, and then, for any sub-audio file, converts the sub-audio file into at least two text data, extracts key nouns of each text data, and determines the first image of each key noun according to each key noun; further, generates a target video according to the first image of each key noun corresponding to each sub-audio file, each sub-audio file and each text data corresponding to each sub-audio file.

[0044] In the present invention, the audio files to be processed that need to be equipped with videos are first segmented, and then speech-to-text conversion and key noun extraction are performed on each sub-audio file, and a corresponding first image is generated according to the key noun. Further, a target video is generated according to the first image of each key noun corresponding to each sub-audio file, each sub-audio file, and each text data corresponding to each sub-audio file. In the present invention, key nouns are first extracted based on the text data converted from the sub-audio files, and then the first image is generated based on the key noun. The process of generating images is simple, thereby improving the efficiency of video generation.

[0045] According to a method for generating video from audio provided by the present invention, key terms of each text data are extracted, including: Segmenting each text data to obtain a segmentation result corresponding to each file data; the segmentation result includes at least one word; Perform part-of-speech tagging on each word to obtain the part-of-speech tagging results of each word; According to the part-of-speech tagging results of each word, nouns in each text data are extracted; The nouns in each text data are prioritized to obtain the key nouns of each text data; the priority of each noun is used to represent the priority of the matching degree of the text data represented by each noun.

[0046] Specifically, in some embodiments, the process of extracting key nouns from each text data in step 102 may be implemented by the following steps, including: It is understandable that key noun extraction usually refers to extracting nouns or noun phrases with important information from text. Noun phrases are usually composed of one or more nouns and can be extracted by analyzing the results of part-of-speech tagging. In Chinese text processing, it usually involves word segmentation and identification of noun entities.

[0047] First, each text data is segmented to obtain the segmentation results corresponding to each file data; the segmentation results include at least one word, and then, each word is tagged with parts of speech to obtain the part-of-speech tagging results of each word. For example, first install the Jieba library and use the Jieba library's application programming interface API, where Jieba is a Chinese text segmentation library that supports three segmentation modes: precise mode: attempts to segment the sentence most accurately, suitable for text analysis; full mode: scans all words in the sentence that can form words, which is very fast, but cannot guarantee the accuracy of each word; search engine mode: based on the precise mode, long words are segmented again to improve the recall rate, which is suitable for search engine indexing. Jieba also supports functions such as keyword extraction and part-of-speech tagging. It uses the keyword extraction function based on the TF-IDF algorithm and the TextRank algorithm to automatically extract keywords from text.

[0048] Exemplarily, in this step, for example, the "jieba.cut" method is used to segment the text and then perform part-of-speech tagging. For another example, the "jieba.posseg.cut" method can be directly used to perform word segmentation and part-of-speech tagging at the same time, and then, jieba is used in combination with part-of-speech tagging to identify nouns.

[0049] Furthermore, nouns in each text data are extracted based on the part-of-speech tagging results of each word. For example, jieba is used to extract the recognized nouns.

[0050] Furthermore, the nouns in each text data are prioritized to obtain the key nouns of each text data. The priority of each noun is used to represent the priority of the matching degree of each noun in representing the text data. The higher the priority of the noun, the higher the matching degree of the name in representing the text data. Correspondingly, the lower the priority of the noun, the lower the matching degree of the name in representing the text data. By prioritizing each extracted noun, and then determining the noun corresponding to the highest priority as the key noun according to the sorting result, the extraction of key nouns of the text data is realized, which facilitates the subsequent image determination of the text data based on the key nouns.

[0051] The method provided in this embodiment first performs word segmentation on each text data to obtain a word segmentation result corresponding to each file data, wherein the word segmentation result includes at least one word; then, performs part-of-speech tagging on each word to obtain a part-of-speech tagging result of each word, and extracts nouns in each text data according to the part-of-speech tagging result of each word; then, the nouns in each text data are prioritized to obtain key nouns of each text data, wherein the priority of each noun is used to characterize the priority of the matching degree of each noun representing the text data; the key nouns of each text data extracted in the present invention characterize that the matching degree of each text data is high; the image of the text data is determined based on the key nouns, the image is determined with high accuracy, the target video generated subsequently is more closely matched with the audio file, and the user experience is better.

[0052] According to a method for generating video from audio provided by the present invention, determining a first image of each key noun according to each key noun includes: According to each key noun, a first prompt word for generating an image is determined; the first prompt word is an input instruction of a large image model; the large image model is obtained by pre-training the image model using a labeled data set; the labeled data set includes a plurality of second prompt words for generating an image and a labeled image generated based on each second prompt word; According to the first prompt word, the first image of each key noun is generated using the image model.

[0053] Specifically, in some embodiments, the process of determining the first image of each key noun according to the key noun of each text data in step 102 can be implemented by the following steps, including: First, according to each key noun, determine the first prompt word for generating the image. Among them, large models (LargeModels) usually refer to machine learning models with a large number of parameters, especially in the field of deep learning. Because these models have many parameters, they can learn complex patterns and features, so large models perform well in various tasks, such as image recognition, natural language processing, games, etc. The large image model is obtained by pre-training the image model using large-scale annotated data. The large-scale annotated data is multiple second prompt words and the annotated images corresponding to each second prompt word. The large image model is for example the range model modelscope or the stable diffusion model in mmagic.

[0054] Prompts are an important concept when interacting with or using large image models. Here are some types of prompts related to large models: Task prompts: explicitly tell the model the type of task to perform, such as translation, summarization, question answering, etc. Format prompts: specify the format of input and output, such as object notation JSON, table, or plain text. Context prompts: provide relevant background information or context to help the model better understand the problem. Example prompts: give one or more examples to show the expected input and output. Constraint prompts: set restrictions on model output, such as word limit, content limit, etc. Style prompts: specify the style of output, such as formal, informal, humorous, etc. Interaction prompts: in conversations or interactive applications, adjust prompts based on previous interactions. Meta prompts: hints about how to use prompts to help users interact with the model more effectively.

[0055] Further, according to the first prompt word, the image model is used to generate a first image corresponding to the key noun. The first prompt word in this embodiment is an input instruction of the image model, which is used to interact with the image model so that the image model generates a first image corresponding to the key noun. Optionally, the output resolution requirement of the first image can be added to the first prompt word so that the quality of the output image meets the requirement, so as to obtain a high-quality first image.

[0056] The method provided in this embodiment first determines the first prompt word for generating an image according to each key noun, wherein the first prompt word is an input instruction of the image large model, and then generates the first image of each keyword according to the first prompt word using the image large model. In the present invention, since the image large model is obtained by pre-training the image model using a labeled data set, the labeled data set includes multiple second prompt words and labeled images corresponding to each second prompt word, and the first image corresponding to the key noun is determined by interacting with the image large model using the first prompt word according to each key noun, the generated first image has a high correlation with the key noun, and the generated first image has a good effect.

[0057] According to a method for generating video from audio provided by the present invention, determining a first image of each key noun according to each key noun includes: Matching the key nouns in a pre-acquired video file library to obtain at least two second images matching each key noun; According to each second image and a preset rule, the first image of each key noun is determined.

[0058] Specifically, in some embodiments, the process of determining the first image of each key noun according to the key noun of each text data in step 102 can be implemented by the following steps, including: First, the key nouns are matched in the pre-acquired video file library to obtain at least two second images matching each key noun. It can be understood that the second images selected according to the key nouns in the pre-acquired video file library, that is, the general images corresponding to the key nouns, can be applicable to the semantic information corresponding to most audio files.

[0059] Furthermore, after generating each second image corresponding to a key noun, the first image corresponding to each key noun can be determined based on each second image and preset rules. For example, each second image is screened and sorted to obtain a high-quality first image. The preset rule is, for example, that the first image only contains scenery scenes, and does not contain frontal facial images, text, brand names, logos, etc. The first image obtained after screening based on the preset rules is an image that only contains scenery scenes, and has a high degree of association with the key noun. For another example, the second image with the highest resolution is determined as the first image, and the first images corresponding to each key noun obtained based on the resolution sorting have a higher image quality.

[0060] The method provided in this embodiment first obtains at least two second images corresponding to each key noun according to the second video file, and then determines the first image of each key noun according to each second image and a preset rule. The process of obtaining the first image based on the method of this embodiment is relatively simple, the image quality of the first image of the key noun obtained based on this method is high, and the effect of generating the target video based on the first image is good and efficient.

[0061] According to a method for generating video from audio provided by the present invention, the first image of each key noun is determined according to each third image and a preset rule, including: Screening the second images according to a preset rule to obtain third images that meet the preset rule; Sorting the resolution of each third image to obtain a resolution sorting result of each third image; According to the resolution sorting results of each third image, the first image of each key noun is obtained.

[0062] Specifically, in some embodiments, the process of determining the first image of each key noun according to each third image and a preset rule can be implemented by the following steps, including: First, the second images are screened according to the preset rules to obtain the third images that meet the preset rules. The preset rules are, for example, that the third image only contains the scene, and does not contain the front face image, text, brand name, logo, etc. The third image obtained after screening based on the preset rules is an image that only contains the scene, and has a high degree of association with the key noun. Other preset rules can also be applied to the present invention.

[0063] Further, the resolution of each third image is sorted to obtain the resolution sorting result of each third image, and the first image of each key noun is obtained according to the resolution sorting result of each third image. For example, the third image with the highest resolution is determined as the first image. It can be understood that the image quality of the first image corresponding to each key noun obtained after the resolution sorting of each third image is higher.

[0064] The method provided in this embodiment firstly screens each second image according to a preset rule to obtain each third image that meets the preset rule, then sorts each third image by resolution to obtain a resolution sorting result of each third image; and then obtains the first image of each key noun according to the resolution sorting result of each third image. The first image of the key noun obtained by this method in this embodiment has a higher image quality, and the effect of generating the target video based on the first image is better.

[0065] According to a method for generating video from audio provided by the present invention, an audio file to be processed is segmented to obtain at least two sub-audio files, including: Determine at least one speech pause point of the audio file to be processed; Determine each speech pause point as each segmentation point of the audio file to be processed; Based on each segmentation point, the audio file to be processed is segmented into sub-audio files corresponding to each segmentation point; The sub-audio files corresponding to each segmentation point are determined to be at least two sub-audio files.

[0066] Specifically, in some embodiments, step 101 may be implemented in the following manner, including: First of all, it is understandable that pauses in speech are natural phenomena in spoken communication, and the speech pauses in the audio files to be processed are crucial to the fluency, clarity, and effectiveness of the language.

[0067] Here are some key points about pause points in speech: Natural pauses: When speaking, people naturally pause between sentences or phrases to help organize their language and breathe. Emphasis pauses: Sometimes, speakers pause before and after important words or concepts to emphasize their importance. Language structure: Certain language structures, such as lists, clauses, etc., may naturally lead to pauses. Rhythm and rhythm: In poetry and rhythmic speech, pauses are part of the rhythm and rhythm. Listening comprehension: Listeners may need time to process and understand the speaker's message, and appropriate pauses can help them follow better. Technical pauses: During a phone or video call, unnatural pauses may occur due to delays or technical issues.

[0068] Based on this, when performing audio segmentation, it can be performed based on voice pause points, first determining at least one voice pause point of the audio, and then determining each voice pause point as each segmentation point of the audio. Further, based on each segmentation point, the audio is segmented into sub-audio files corresponding to each segmentation point.

[0069] For example, there are 5 speech pause points in the audio file to be processed. Based on the 5 speech pause points, the audio file to be processed can be divided into 6 sub-audio files, and the 6 sub-audio files can be processed in sequence subsequently.

[0070] Finally, the sub-audio files corresponding to each segmentation point can be determined as multiple sub-audio files obtained by segmentation. Subsequently, speech-to-text conversion, key noun extraction, corresponding image generation, etc. can be performed for each sub-audio file, which can improve the accuracy of video generation to a certain extent.

[0071] The method provided in this embodiment first determines at least one speech pause point of the audio, and determines each speech pause point as each segmentation point of the audio; then, based on each segmentation point, the audio is segmented into sub-audio files corresponding to each segmentation point; and then, the sub-audio files corresponding to each segmentation point are determined as multiple sub-audio files. In the method provided in this embodiment, audio segmentation is performed based on speech pause points, and the segmentation effect is good, which conforms to the human auditory habits. Then, for each segmented sub-audio, speech-to-text conversion, key noun extraction, image generation, and video generation are performed, and the audio content and video content of the target video obtained can synchronously convert scenes, and the viewing effect is good.

[0072] According to a method for generating video from audio provided by the present invention, a first sub-audio video file corresponding to the sub-audio file is generated according to the first image of each key noun, the sub-audio file and the corresponding text data, including: synthesizing the first image of each key noun with each sub-audio file to obtain a second sub-audio video file corresponding to each sub-audio file; According to each second sub-audio and video file and each text data corresponding to each sub-audio file, a first sub-audio and video file corresponding to each sub-audio file is obtained.

[0073] Specifically, in some embodiments, the process of generating the first sub-audio and video files corresponding to each sub-audio file in step 104 can be implemented by the following steps, including: First, the first image of each key noun corresponding to each sub-audio file is synthesized with each sub-audio file to obtain a second sub-audio and video file corresponding to each sub-audio file, wherein the frame rate of the second sub-audio and video file can be set to 16, the resolution is customized, the video encoding format defaults to the moving image compression MPEG format, and the audio encoding format remains in the original format.

[0074] Furthermore, the first sub-audio and video files corresponding to each sub-audio file are obtained according to each second sub-audio and video file and each text data corresponding to each sub-audio file. In order to ensure time synchronization and to ensure that the final synthesized target video contains subtitles, each text data (i.e., each text record) corresponding to each sub-audio file can be burned into each sub-audio and video file, so as to facilitate the synchronous display of subtitles while playing the target video later.

[0075] The method provided in this embodiment first synthesizes the first image of each key noun corresponding to each sub-audio file with each sub-audio file to obtain the second sub-audio and video file corresponding to each sub-audio file, and then obtains the first sub-audio and video file corresponding to each sub-audio file according to each second sub-audio and video file and each text data corresponding to each sub-audio file. In this embodiment, the first image and the corresponding sub-audio file are merged and encoded into a sub-audio and video file, and all the sub-audio and video files are merged into a whole video file, so as to facilitate the final merge and output of the target video. The target video file merged and output by this method has no audio and video asynchronism, and the audio content and video content can synchronously switch scenes, which provides a good user experience.

[0076] According to a method for generating video from audio provided by the present invention, obtaining first sub-audio and video files corresponding to each sub-audio file according to each second sub-audio and video file and each text data corresponding to each sub-audio file includes: Encoding each second sub-audio and video file to obtain each encoded second sub-audio and video file; According to the encoded second sub-audio and video files and the text data corresponding to the sub-audio files, the first sub-audio and video files corresponding to the sub-audio files are obtained.

[0077] Specifically, in some embodiments, the process of determining the first sub-audio and video files corresponding to each sub-audio file may be implemented by the following steps, including: First, each second sub-audio and video file is encoded to obtain the encoded second sub-audio and video files, wherein each second sub-audio and video file is sequentially encoded to obtain the encoding order of each second sub-audio and video file, which is convenient for subsequent merging according to the encoding order to obtain the target video.

[0078] Furthermore, the first sub-audio and video files corresponding to each sub-audio file are obtained according to each encoded second sub-audio and video file and each text data corresponding to each sub-audio file. In order to ensure time synchronization and to ensure that the final synthesized target video contains subtitles, each text data (i.e., each text record) corresponding to each encoded sub-audio file can be burned into each encoded sub-audio and video file, so as to facilitate the synchronous display of subtitles while playing the target video later.

[0079] The method provided in this embodiment first encodes the second sub-audio and video files corresponding to each sub-audio file, and then obtains the first sub-audio and video files corresponding to each sub-audio file based on the encoded second sub-audio and video files and the text data corresponding to each sub-audio file, that is, adding subtitles to the target video can improve the user's viewing experience.

[0080] According to a method for generating video from audio provided by the present invention, encoding each second sub-audio and video file to obtain each encoded second sub-audio and video file includes: Get the video encoding format, key frame interval and audio encoding format; According to the video encoding format, the key frame interval and the audio encoding format, the second sub-audio and video files are sequentially encoded to obtain the encoded second sub-audio and video files.

[0081] Specifically, in some embodiments, the process of encoding the second sub-audio and video files corresponding to each sub-audio file to obtain the encoded second sub-audio and video files can be implemented by the following steps, including: First, obtain the video encoding format, key frame interval, and audio encoding format.

[0082] Among them, video coding formats are a series of standards and methods used to compress video data into files. These formats usually include specific compression algorithms and encoding technologies to reduce the size of video files while trying to maintain video quality. The following are some common video coding formats: 1. Advanced Video Coding (H.264 / AVC): A widely used encoding format that supports high-definition video and is widely used in online video, Blu-ray discs, mobile video, etc.

[0083] 2. High Efficiency Video Coding (H.265 / HEVC): It is the successor of H.264 and provides higher compression efficiency for 4K and 8K videos.

[0084] 3. VP9: An open source coding format used for online videos, especially YouTube.

[0085] 4. AV1: An open source coding format developed by the Alliance for Open Media (AOMedia) to provide higher compression efficiency than H.265.

[0086] 5. MPEG-4 Part 2: Mainly used for video calls and video conferencing, supports lower resolution videos.

[0087] Exemplarily, the video encoding format defaults to Motion Picture Experts Group (MPEG).

[0088] Among them, audio coding formats are used to compress audio data into files to reduce the requirements of storage space and transmission bandwidth. The following are some common audio coding formats: 1. MP3 audio file format (MPEG-1 Audio Layer 3, MP3): a widely used lossy compression audio encoding format, known for its small file size and good sound quality.

[0089] 2. Advanced Audio Coding (AAC): Developed from the MPEG-4 standard, it provides better sound quality and higher compression efficiency than MP3.

[0090] 3. Advanced Video Coding Ogg Vorbis: An open source audio coding format that is popular for its good sound quality and openness.

[0091] Exemplarily, the audio encoding format in this embodiment maintains the original format of the audio file to be processed.

[0092] Among them, a key frame is a fully encoded frame in a video sequence, which contains all the information of a frame and can therefore be decoded independently. The frames between key frames are called P frames (Predicted frames) or B frames (Bidirectional predicted frames), which only store the difference information with the key frame or the previous and next frames to reduce the amount of data. For example, the frame rate is set to 16. Other parameters such as resolution can be selected as needed.

[0093] Furthermore, the second sub-audio and video files corresponding to each sub-audio file can be sequentially encoded according to the video encoding format, the key frame interval, and the audio encoding format to obtain the encoded second sub-audio and video files, and the encoded second sub-audio and video files include the encoding sequence of the second sub-audio and video files. For example, the encoding sequence of the second sub-audio and video files is 001, 002, 003, 004, 005, 006, 007, 008, 009, and 010 in chronological order.

[0094] The method provided in this embodiment first obtains the video encoding format, key frame interval and audio encoding format, and sequentially encodes the second sub-audio and video files corresponding to each sub-audio file according to the video encoding format, key frame interval and audio encoding format to obtain the encoded second sub-audio and video files. The encoded second sub-audio and video files include the encoding order of the second sub-audio and video files, which is convenient for subsequent sub-audio and video merging based on the encoding order.

[0095] According to a method for generating video from audio provided by the present invention, first sub-audio and video files corresponding to each sub-audio file are merged to obtain a target video, including: In the case where there are multiple first sub-audio and video files, the encoding order of the multiple first sub-audio and video files is obtained; the encoding order of the multiple first sub-audio and video files is consistent with the encoding order of the encoded second sub-audio and video files; The plurality of first sub-audio and video files are merged according to the encoding order to obtain the target video.

[0096] Specifically, in some embodiments, the process of merging the first sub-audio and video files to obtain the target video can be implemented by the following steps, including: For example, each first sub-audio and video file is merged according to the time point of each sub-audio file to obtain a complete target video. Specifically, in the case where there are multiple first sub-audio and video files, the encoding order of each encoded first sub-audio and video file is obtained. It can be understood that each encoded first sub-audio and video file corresponds to its own encoding order, and its encoding order is used to characterize the sequential time points of each first sub-audio and video, that is, the sequential position in the target video. For example, the encoding order of the first sub-audio and video is 001, 002, 003, 004, 005, 006, 007, 008, 009, and 010 in time order.

[0097] Furthermore, the first sub-audio and video files are merged according to the encoding order of the first sub-audio and video files, so as to obtain the target video and realize video generation.

[0098] The method provided in this embodiment first obtains the encoding order of each encoded first sub-audio and video file, and then merges each first sub-audio and video file according to the encoding order to obtain the target video, thereby realizing video generation. The video file merged and output by the method provided in this embodiment has no audio and video asynchronism, and the audio content and the video content can synchronously switch scenes.

[0099] Figure 2 FIG. 2 is a flow chart of the method for generating video from audio provided by the present invention. Figure 2 As shown, the method comprises the following steps: Step 201: Segment the audio.

[0100] Step 202: Convert speech to text.

[0101] Step 203: extract key nouns.

[0102] Step 204: Generate images corresponding to the key nouns.

[0103] Step 205: Encode and synthesize the sub-audio and video files.

[0104] Step 206: Merge the sub-audio and video files.

[0105] Figure 3 Schematic diagram of the structure of the video generation system provided by the present invention. Figure 3 As shown, the video generation system 300 includes: an audio segmentation component 310, a text processing component 320, a picture generation component 330 and a video merging component 340.

[0106] The video generating device provided by the present invention is described below. The video generating device described below and the method for generating video from audio described above can be referenced to each other.

[0107] Figure 4 Schematic diagram of the structure of the device for generating video from audio provided by the present invention, such as Figure 4 As shown, the apparatus 400 for generating video from audio includes the following modules: An audio segmentation module 410 is used to segment the audio file to be processed to obtain at least two sub-audio files; The image generation module 420 is used to convert any of the sub-audio files into at least two text data; extract key nouns from each of the text data, and determine the first image of each of the key nouns according to each of the key nouns; The video generation module 430 is used to generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

[0108] The device provided in this embodiment divides the audio file to be processed through the audio segmentation module 410 to obtain multiple sub-audio files. Then, the image generation module 420 converts the sub-audio file into at least two text data for any sub-audio file, extracts the key nouns of each text data, and determines the first image of each key noun according to each key noun; further, the video generation module 430 generates a first sub-audio and video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data, and merges the first sub-audio and video files corresponding to each sub-audio file to obtain the target video.

[0109] In the present invention, the audio files to be processed that need to be equipped with videos are first segmented, and then each sub-audio file is respectively converted from speech to text, key nouns are extracted, and a corresponding first image is generated according to the key nouns. Furthermore, according to the first images of each key noun corresponding to each sub-audio file, the sub-audio files and each text data corresponding to the sub-audio files, the first sub-audio and video files corresponding to the sub-audio files are generated, and each first sub-audio and video file is merged to obtain a target video. In the present invention, the image generation module 420 first extracts key nouns based on the text data converted from the sub-audio files, and then generates a first image based on the key nouns. The process of generating images is simple, thereby improving the efficiency of video generation and achieving synchronized audio and video display in the generated target video.

[0110] According to the apparatus 400 for generating video from audio provided by the present invention, the image generating module 420 is specifically used for: Performing word segmentation on each of the text data to obtain a word segmentation result corresponding to each of the file data; the word segmentation result includes at least one word; Performing part-of-speech tagging on each of the words to obtain part-of-speech tagging results for each of the words; Extracting nouns from the text data according to the part-of-speech tagging results of the words; The nouns in each of the text data are prioritized to obtain key nouns in each of the text data; the priority of the noun is used to represent the priority of the matching degree of the noun representing the text data.

[0111] According to the apparatus 400 for generating video from audio provided by the present invention, the image generating module 420 is further used for: According to each of the key nouns, a first prompt word for generating an image is determined; the first prompt word is an input instruction of a large image model; the large image model is obtained by pre-training the image model using a labeled data set; the labeled data set includes a plurality of second prompt words for generating an image and labeled images generated based on each of the second prompt words; According to the first prompt word, the first image of each of the key nouns is generated using the image macromodel.

[0112] According to the apparatus 400 for generating video from audio provided by the present invention, the image generating module 420 is further used for: Matching each of the key nouns in a pre-acquired video file library to obtain at least two second images matching each of the key nouns; According to each of the second images and a preset rule, a first image of each of the key nouns is determined.

[0113] According to the apparatus 400 for generating video from audio provided by the present invention, the image generating module 420 is further used for: Screening each of the second images according to the preset rule to obtain each of the third images that meets the preset rule; Sorting the resolution of each of the third images to obtain a resolution sorting result of each of the third images; The first image of each of the key nouns is obtained according to the resolution sorting result of each of the third images.

[0114] According to the apparatus 400 for generating video from audio provided by the present invention, the audio segmentation module 410 is specifically used for: Determine at least one speech pause point of the audio file to be processed; Determine each of the speech pause points as each segmentation point of the audio file to be processed; Based on each of the segmentation points, the audio file to be processed is segmented into sub-audio files corresponding to each of the segmentation points; The sub-audio files corresponding to the segmentation points are determined as the at least two sub-audio files.

[0115] According to the apparatus 400 for generating video from audio provided by the present invention, the video generating module 430 is specifically used for: synthesizing the first image of each of the key nouns with each of the sub-audio files to obtain a second sub-audio video file corresponding to each of the sub-audio files; According to each of the second sub-audio and video files and each of the text data corresponding to each of the sub-audio files, a first sub-audio and video file corresponding to each of the sub-audio files is obtained.

[0116] According to the apparatus 400 for generating video from audio provided by the present invention, the video generating module 430 is further used for: Encoding each of the second sub-audio and video files to obtain encoded each of the second sub-audio and video files; According to the encoded second sub-audio and video files and the text data corresponding to the sub-audio files, the first sub-audio and video files corresponding to the sub-audio files are obtained.

[0117] According to the apparatus 400 for generating video from audio provided by the present invention, the video generating module 430 is further used for: Get the video encoding format, key frame interval and audio encoding format; According to the video encoding format, the key frame interval and the audio encoding format, the second sub-audio and video files are sequentially encoded to obtain the encoded second sub-audio and video files.

[0118] According to the apparatus 400 for generating video from audio provided by the present invention, the video generating module 430 is further used for: In the case where there are multiple first sub-audio and video files, obtaining the encoding order of the multiple first sub-audio and video files; the encoding order of the multiple first sub-audio and video files is consistent with the encoding order of the encoded second sub-audio and video files; The plurality of the first sub-audio and video files are merged according to the encoding order to obtain the target video.

[0119] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communications interface 520 and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the method for generating video from audio, which includes: Splitting the audio file to be processed to obtain at least two sub-audio files; For any of the sub-audio files, convert the sub-audio file into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; Generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

[0120] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0121] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the method of generating video from audio provided by the above methods, the method includes: Splitting the audio file to be processed to obtain at least two sub-audio files; For any of the sub-audio files, convert the sub-audio file into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; Generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

[0122] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for generating a video from audio provided by the above methods is implemented, and the method comprises: Splitting the audio file to be processed to obtain at least two sub-audio files; For any of the sub-audio files, convert the sub-audio file into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; Generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

[0123] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0124] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating video from audio, It is characterized in that include: Splitting the audio file to be processed to obtain at least two sub-audio files; For any of the sub-audio files, convert the sub-audio file into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; Generate a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

2. The method for generating video from audio according to claim 1, It is characterized in that The extracting of key nouns from each of the text data includes: Performing word segmentation on each of the text data to obtain a word segmentation result corresponding to each of the file data; the word segmentation result includes at least one word; Performing part-of-speech tagging on each of the words to obtain part-of-speech tagging results for each of the words; Extracting nouns from the text data according to the part-of-speech tagging results of the words; The nouns in each of the text data are prioritized to obtain key nouns in each of the text data; the priority of the noun is used to represent the priority of the matching degree of the noun representing the text data.

3. The method for generating video from audio according to claim 1, It is characterized in that The step of determining the first image of each key noun according to each key noun includes: According to each of the key nouns, a first prompt word for generating an image is determined; the first prompt word is an input instruction of a large image model; the large image model is obtained by pre-training the image model using a labeled data set; the labeled data set includes a plurality of second prompt words for generating an image and labeled images generated based on each of the second prompt words; According to the first prompt word, the first image of each of the key nouns is generated using the image macromodel.

4. The method for generating video from audio according to claim 1, It is characterized in that The step of determining the first image of each key noun according to each key noun includes: Matching each of the key nouns in a pre-acquired video file library to obtain at least two second images matching each of the key nouns; According to each of the second images and a preset rule, a first image of each of the key nouns is determined.

5. The method for generating video from audio according to claim 4, It is characterized in that The step of determining the first image of each key noun according to each second image and a preset rule includes: Screening each of the second images according to the preset rule to obtain each of the third images that meets the preset rule; Sorting the resolution of each of the third images to obtain a resolution sorting result of each of the third images; The first image of each of the key nouns is obtained according to the resolution sorting result of each of the third images.

6. The method for generating video from audio according to any one of claims 1 to 5, It is characterized in that The step of dividing the audio file to be processed to obtain at least two sub-audio files includes: Determine at least one speech pause point of the audio file to be processed; Determine each of the speech pause points as each segmentation point of the audio file to be processed; Based on each of the segmentation points, the audio file to be processed is segmented into sub-audio files corresponding to each of the segmentation points; The sub-audio files corresponding to the segmentation points are determined as the at least two sub-audio files.

7. The method for generating video from audio according to any one of claims 1 to 5, It is characterized in that The step of generating a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data includes: synthesizing the first image of each of the key nouns with each of the sub-audio files to obtain a second sub-audio video file corresponding to each of the sub-audio files; According to each of the second sub-audio and video files and each of the text data corresponding to each of the sub-audio files, a first sub-audio and video file corresponding to each of the sub-audio files is obtained.

8. The method for generating video from audio according to claim 7, It is characterized in that The obtaining, according to each of the second sub-audio and video files and each of the text data corresponding to each of the sub-audio files, a first sub-audio and video file corresponding to each of the sub-audio files comprises: Encoding each of the second sub-audio and video files to obtain encoded each of the second sub-audio and video files; According to the encoded second sub-audio and video files and the text data corresponding to the sub-audio files, the first sub-audio and video files corresponding to the sub-audio files are obtained.

9. The method for generating video from audio according to claim 8, It is characterized in that The step of encoding each of the second sub-audio and video files to obtain each encoded second sub-audio and video file includes: Get the video encoding format, key frame interval and audio encoding format; According to the video encoding format, the key frame interval and the audio encoding format, the second sub-audio and video files are sequentially encoded to obtain the encoded second sub-audio and video files.

10. The method of generating video from audio according to claim 1, It is characterized in that The step of merging the first sub-audio and video files corresponding to the sub-audio files to obtain a target video includes: In the case where there are multiple first sub-audio and video files, obtaining the encoding order of the multiple first sub-audio and video files; the encoding order of the multiple first sub-audio and video files is consistent with the encoding order of the encoded second sub-audio and video files; The plurality of the first sub-audio and video files are merged according to the encoding order to obtain the target video.

11. A device for generating video from audio, It is characterized in that include: An audio segmentation module, used for segmenting the audio file to be processed to obtain at least two sub-audio files; An image generation module, used for converting any of the sub-audio files into at least two pieces of text data; Extracting key nouns from each of the text data, and determining a first image of each of the key nouns according to each of the key nouns; A video generation module, used for generating a first sub-audio video file corresponding to the sub-audio file according to the first image of each key noun, the sub-audio file and the corresponding text data; The first sub-audio and video files corresponding to the sub-audio files are merged to obtain a target video.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method for generating video from audio according to any one of claims 1 to 10 is implemented.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for generating video from audio according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the method for generating video from audio according to any one of claims 1 to 10 is implemented.

Images