Method, device, electronic device and computer-readable medium for generating video from speech

Abstract

The present disclosure provides a method, device, electronic device and computer-readable medium for generating video by voice, and relates to the field of video production technology. The method for generating video by voice includes: obtaining a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction; obtaining voice information; identifying the semantics corresponding to the voice information according to the voice information; and generating a first type of video according to the semantics. The technical solution provided by the present disclosure obtains voice information only after obtaining the voice function activation instruction, which can prevent the corresponding function from being enabled when the video generation is not needed. The present disclosure automatically generates a video according to the semantics, and the generated video meets the needs of the user. The user does not need to manually search for video materials or manually make videos by himself. The video generation is convenient and fast, and the video generation efficiency is high.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of video production. Specifically, the present disclosure relates to a method, device, electronic device and computer-readable medium for generating video through speech. Background Art

[0002] With the development of the Internet and smart terminals, more and more users use terminals to make videos and share them with others to gain attention, clicks or fans, etc. For example, on a short video viewing platform, users will post short videos to publish them for others to watch.

[0003] To publish a video, you need to make a video. In the prior art, when a user makes a video, he needs to make the video manually, such as manually shooting the video, or manually finding video materials, and then making the video. It takes a lot of time to make a video, and the efficiency of making a video is low. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, a method for generating a video from speech is provided, the method comprising:

[0006] Obtain a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction;

[0007] Get voice information;

[0008] Identify semantics corresponding to the voice information according to the voice information;

[0009] A first type of video is generated according to the semantics.

[0010] In a second aspect, a device for generating video by voice is also provided, the device comprising:

[0011] An instruction acquisition module, used to acquire a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction;

[0012] A voice acquisition module is used to acquire voice information;

[0013] A speech recognition module, used for recognizing the semantics corresponding to the speech information according to the speech information;

[0014] A video generation module is used to generate a first type of video according to the semantics.

[0015] In a third aspect, an electronic device is also provided, the electronic device comprising:

[0016] one or more processors;

[0017] Memory;

[0018] One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more programs are configured to: execute the method of generating video from speech shown in the first aspect of the present disclosure.

[0019] In a fourth aspect, a computer-readable medium is also provided, on which a computer program is stored, and when the program is executed by a processor, the method of generating video by voice shown in the first aspect of the present disclosure is implemented.

[0020] Compared with the prior art, the embodiments of the present disclosure provide a method, device, electronic device and computer-readable medium for generating video from voice. The voice information is obtained only after the voice function activation instruction is obtained, which can prevent the corresponding function from being enabled when the video generation is not needed. The present disclosure automatically generates video according to semantics. The video is not only related to the keywords or key words in the voice information, but is related to the entire semantics corresponding to the voice information. The generated video meets the usage scenario expressed by the user semantics and meets the various needs of the user semantics. The user does not need to manually search for video materials or manually make videos. The video generation is convenient and fast, and the video generation efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of a method for generating video from speech provided in an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of the position of a virtual control A provided in an embodiment of the present disclosure on a display interface of a terminal;

[0024] Figure 3 for Figure 1 Detailed flow chart of step S104;

[0025] Figure 4 A schematic diagram of an interface for displaying voice prompt information provided in an embodiment of the present disclosure;

[0026] Figure 5A method for generating video from speech provided by an embodiment of the present disclosure also includes a flowchart;

[0027] Figure 6 A schematic diagram of the structure of a device for generating video by voice provided in an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the structure of an electronic device for generating video by voice provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish between devices, modules or units, and are not used to limit these devices, modules or units to be different devices, modules or units, nor are they used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] The method, device, electronic device and medium for generating video from speech provided in the present disclosure are intended to solve the above technical problems in the prior art.

[0036] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0037] Those skilled in the art can understand that the “terminal” used in the present disclosure may be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a MID (Mobile Internet Device), etc.

[0038] See also Figure 1 In an embodiment of the present disclosure, a method for generating video by voice is provided. The method for generating video by voice can be applied to a terminal. The method includes:

[0039] Step S101: Acquire a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction.

[0040] See also Figure 2 , the voice function activation instruction can be triggered by the user operating the terminal. Specifically, the voice function activation instruction can be triggered by the user pressing the preset virtual control for a preset time. The preset time is not limited, such as the preset time can be 1s, 2s or 2.5s, etc. When the preset time is 2s, if the user presses the virtual control for greater than or equal to 2s, the terminal can obtain the voice function activation instruction. Specifically, such as an app (Application, mobile phone software) for shooting short videos, there is a virtual control A in the lower middle part of the display interface of the app. The user presses the virtual control A with his finger for more than the preset time, and the terminal can obtain the voice function activation instruction. After the terminal turns on the voice function, the terminal can obtain voice information.

[0041] Step S102: Acquire voice information.

[0042] After the terminal turns on the voice function, if the user inputs voice information, that is, the user speaks, the terminal can obtain the voice information corresponding to the user's speech. For example, if the user's words are "I want to make a card point video of my trip to Chongqing", the terminal can obtain the voice information of "I want to make a card point video of my trip to Chongqing".

[0043] Step S103: Identify the semantics corresponding to the voice information according to the voice information.

[0044] If the voice message is "I want to make a video of my trip to Chongqing", the terminal will parse the semantics of the voice message to obtain the semantics of the voice message, and the terminal can determine what the user wants to do and what type of video is needed. The specific solution for obtaining the semantics corresponding to the voice message is a prior art and will not be described in this disclosure.

[0045] Step S104: Generate a first type of video according to semantics.

[0046] Generating a first type of video according to semantics may include: acquiring video material corresponding to the semantics, and generating the first type of video according to the video material.

[0047] If the voice message is "I want to make a card point video of my trip to Chongqing", the semantics corresponding to the voice message are obtained, and the video material corresponding to the semantics can be obtained locally in the terminal. "I want to make a card point video of my trip to Chongqing", the corresponding video material is the photo of the trip to Chongqing and the audio preset locally in the terminal. When the user takes a photo, the photo obtained may include basic information of the photo, such as the basic information of the photo includes the location information and time information when the photo was taken. The terminal obtains the basic information of the photo, takes the photo whose basic information conforms to the semantics "I want to make a card point video of my trip to Chongqing" as the video material, and obtains the preset audio as the video material, and generates a video based on the photo and the audio. Among them, there can be multiple preset audios. When selecting one of the audios as the video material, you can randomly select an audio as the video material, or select an audio that conforms to the semantics as the video material. Specifically, if the user's mood is judged according to the semantics, and the user's mood is cheerful, then the cheerful type of audio is selected as the video material.

[0048] The method for generating video from voice provided by the embodiment of the present disclosure obtains voice information only after obtaining the voice function activation instruction, which can prevent the corresponding function from being enabled when there is no need to generate a video. The present disclosure automatically generates video according to semantics. The video is not only related to the keywords or key words in the voice information, but is related to the entire semantics corresponding to the voice information. The generated video meets the usage scenario expressed by the user semantics, and the generated video meets the various needs of the user semantics. The user does not need to manually search for video materials or manually make videos. The video generation is convenient and fast, and the video generation efficiency is high.

[0049] See also Figure 3 , optionally, generating a first type of video according to semantics, including:

[0050] S301: Determine video dimensions of a generated video according to semantics, where the video dimensions include at least two of time, location, person, scenic spot, emotion, speech-to-text effect, audio, scene, and special effect.

[0051] In the present disclosure, video temperature includes but is not limited to time, place, person, object, scenic spot, emotion, speech-to-text effect, audio, scene, and special effects.

[0052] Among them, the time, place and person can be the time, place and person corresponding to the video material, and the object can be the object of the video material, such as a photo. The speech-to-text effect can be the letter effect in the produced video. The scene can be the scene of the material. For example, if the object is a photo and the scene is a grassland, then the corresponding grassland photo is obtained. The special effect can be the effect added to the video, such as applying a filter. After obtaining the voice corresponding to the voice information, the dimension of the video can be determined. For example, if the voice information is "Help me make a photo movie with the photos of cats taken yesterday", after determining the semantics of the voice information, the dimensions of the video can be determined according to the voice, including the time "yesterday", the person "cat" and the object "photo".

[0053] S302: Obtain video material according to video dimensions.

[0054] The video material is pre-stored in the terminal. The video material can be collected and generated by the user in his daily life, such as photos taken by the user himself. The type of video material is not limited, such as photos, audio and text. The video material includes preset basic information. The basic information of the video material corresponds to the video dimension. After the video dimension is determined, the video material can be obtained according to the video dimension. If the video dimension includes the time "yesterday", the person "cat" and the object "photo", it is necessary to obtain the photo of the cat taken yesterday, so as to use the obtained photo as the video material, that is, the video material corresponding to the video dimension is obtained according to the basic information corresponding to the video dimension and the video material.

[0055] Optionally, a preset video material, such as a preset audio, may also be obtained. There may be multiple preset audios, and when one of the audios is selected as the video material, an audio may be randomly selected as the video material, or an audio that meets the semantics may be selected as the video material.

[0056] S303: Generate a first type of video according to the video material and semantics.

[0057] After the video material is obtained, the first type of video can be generated according to the video material and semantics.

[0058] For example, if the voice message is "Please help me make a photo movie with my photos taken yesterday, and add special effect A to my image", then when the video dimensions are obtained as time "yesterday", person "me", and object "photo", the corresponding material is obtained according to the video dimensions, that is, my photos taken yesterday, and in the generated video, special effect A is added to the image of "me".

[0059] The solution of the embodiment of the present disclosure takes into account at least two video dimensions when generating a video, and the generated video meets the user's needs.

[0060] Optionally, after obtaining the voice function activation instruction, the method for generating a video by voice further includes:

[0061] Display voice prompt information, the voice prompt information is used to prompt the user to input voice information according to the voice prompt information.

[0062] See also Figure 4 After watching the voice prompt information, the user can input voice information according to the voice prompt information. The voice prompt information provides a reference for the user, prompting the user to input voice information according to the preset format. The voice prompt information is used to prompt the user to input voice information according to the voice prompt information, and the voice prompt information can be used to prompt the user to input voice information in the format of the voice prompt information. When the user inputs voice information according to the voice prompt information, it can make it easier for the terminal to recognize the user's voice information, so that it is easy to judge the semantics corresponding to the voice information, and the corresponding video material can be quickly obtained. For example, the voice prompt information may be "I want to make a card point video of the trip to Chongqing", then the user can input the voice information "I want to make a card point video of the trip to Beijing", if the user does not input the voice information according to the voice prompt information, but speaks casually, then the corresponding first type of video may not be generated according to the voice information.

[0063] The specific type of the voice prompt information is not limited. Optionally, the voice prompt information may include multiple prompt information for enabling the terminal to perform different functions. Optionally, the voice prompt information includes first type information and second type information, the first type information is used to generate a first type video, and the second type information is used to implement other functions.

[0064] Optionally, the voice prompt information includes first type information, second type information and third type information, and identifying semantics corresponding to the voice information according to the voice information; and generating a first type video according to the semantics includes:

[0065] When the type of the voice information is the first type of voice, semantics corresponding to the voice information are identified according to the voice information; and a first type of video is generated according to the semantics.

[0066] That is, a voice function activation instruction is obtained to activate the voice function corresponding to the voice function activation instruction and obtain voice information; voice prompt information is displayed, and the voice prompt information is used to prompt the user to input voice information according to the voice prompt information; and when the type of the voice information is the first type of voice, steps S103 and S104 in the present disclosure are executed to generate a first type of video.

[0067] See also Figure 5 After acquiring the voice information, the method for generating a video from voice also includes:

[0068] S501: Determine the type of voice information according to a preset language model corresponding to the voice prompt information, where the types of voice information include first type voice, second type voice and third type voice.

[0069] The language model corresponds to the voice prompt information. The language model can be obtained by training based on multiple preset voice samples. The first type of voice and the second type of voice correspond to multiple voice samples respectively, and the language model is obtained by training based on multiple voice samples corresponding to the first type of voice and the second type of voice respectively. The voice information can be determined to be the first type of voice or the second type of voice based on the preset language model corresponding to the voice prompt information. When the voice information does not belong to the first type of voice or the second type of voice, the voice information is determined to be the third type of voice.

[0070] Optionally, determining the type of the voice information according to a preset language model corresponding to the voice prompt information includes:

[0071] Determining whether the voice information is a first type of voice or a second type of voice according to a preset language model corresponding to the voice prompt information;

[0072] When the voice does not belong to the first type of voice or the second type of voice, the voice information is determined to be the third type of voice.

[0073] When the type of the voice information is the first type of voice, semantics corresponding to the voice information are identified according to the voice information; and a first type of video is generated according to the semantics.

[0074] S502: When the type of the voice information is the second type of voice, enable the function corresponding to the voice information according to the voice information.

[0075] The second type of voice is used to turn on the function of the corresponding voice message. If the voice message is "I want to try the most popular series recently", the terminal turns on the series mode and pins the most popular series recently to the top, so that the user can see the preview of the most popular series; if the voice message is "I want to try the latest props", the terminal turns on the front camera and turns on the latest props, so that the user can use the latest props to take selfies. The specific voice information corresponding to the second type of voice is not limited, and it can enable the terminal to turn on the function corresponding to the voice message. It can be understood that after the terminal turns on the function corresponding to the voice message, the user can directly manually operate the terminal to use the corresponding function, or the user continues to make a voice so that the terminal uses the corresponding function according to the voice information corresponding to the voice. How the user of the terminal uses the corresponding function is not limited in this disclosure.

[0076] S503: When the type of the voice information is the third type of voice, generate a second type of video according to the voice information.

[0077] Optionally, generating the second type of video according to the voice information may include:

[0078] Get the text corresponding to the voice information;

[0079] A second type of video is generated according to the preset audio corresponding to the text and the third type of voice.

[0080] After the voice information is obtained, the text corresponding to the voice information can be parsed. For example, if the voice information corresponding to the third type of voice obtained is "I am very happy today", the text corresponding to the voice information "I am very happy today" can be obtained.

[0081] The third type of voice can be preset to correspond to one or more audios. The audio is music. When generating the second video, one or more audios corresponding to the third type can be selected as video materials to generate the second type of video. It can be understood that when the second video is played, the user can watch the text corresponding to the voice information he said and hear the corresponding audio.

[0082] Optionally, generating the second type of video according to the voice information may include:

[0083] Generate a dynamic popper corresponding to the frequency of the voice information according to the voice information;

[0084] Get the user's preset account avatar;

[0085] The second type of video is generated according to the preset audio corresponding to the dynamic pop, the avatar and the third type of voice.

[0086] The dynamic pop is a dynamic curve that changes with the frequency of the voice information. The voice information is formed by the voice of the user, and the sound has a frequency, so the dynamic pop can be generated.

[0087] The preset account is the account for logging into the application on the terminal. Optionally, the solution of the present disclosure is executed by the application running on the terminal. The user may have an account for the application, and the avatar corresponding to the account is displayed when logging into the account, and the terminal may obtain the avatar of the user's account.

[0088] When generating a video, a second type of video is generated based on the dynamic pop, the avatar and the third type of voice. It can be understood that when the second video is played, the user can watch the dynamic pop corresponding to the voice information he said and the avatar of his account, and can also hear the corresponding audio.

[0089] The above is only an example of generating the second type of video by the third type of speech. In the present disclosure, there is no limitation on how to generate the second type of video according to the third type of speech.

[0090] The technical solution disclosed in the present invention can determine the type of voice information according to the user's voice information to execute different functions. The technical solution disclosed in the present invention diversifies the role of voice, determines the type of voice information according to a preset language model of the corresponding voice prompt information, and recognizes the voice type quickly.

[0091] See also Figure 6 The embodiment of the present disclosure provides a device 60 for generating video by voice, which can implement the method for generating video by voice of the above embodiment. The device 60 for generating video by voice may include:

[0092] The instruction acquisition module 601 is used to acquire a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction;

[0093] A voice acquisition module 602 is used to acquire voice information;

[0094] A speech recognition module 603, used for recognizing the semantics corresponding to the speech information according to the speech information;

[0095] The video generation module 604 is used to generate a first type of video according to semantics.

[0096] The device for generating video from voice provided by the embodiment of the present disclosure obtains voice information only after obtaining the voice function activation instruction, which can prevent the corresponding function from being enabled when there is no need to generate a video. The present disclosure automatically generates video according to semantics. The video is not only related to the keywords or key words in the voice information, but is related to the entire semantics corresponding to the voice information. The generated video meets the usage scenario expressed by the user semantics, and the generated video meets the various needs of the user semantics. The user does not need to manually search for video materials or manually make videos. The video generation is convenient and fast, and the video generation efficiency is high.

[0097] The video generation module 604 may include:

[0098] A dimension acquisition unit, used to determine the video dimension of the generated video according to the semantics, where the video dimension includes at least two of time, place, person, object, scenic spot, emotion, speech-to-text effect, audio, scene, and special effect;

[0099] A material acquisition unit, used to acquire video material according to video dimensions;

[0100] The first video generating unit is used to generate a first type of video according to video material and semantics.

[0101] The device 60 for generating video by voice may further include:

[0102] The prompt display module is used to display voice prompt information, and the voice prompt information is used to prompt the user to input voice information according to the voice prompt information.

[0103] The speech recognition module 603 is specifically used to recognize the semantics corresponding to the speech information according to the speech information when the type of the speech information is the first type of speech.

[0104] The device 60 for generating video by voice may further include:

[0105] A voice type determination module, used to determine the type of voice information according to a preset language model corresponding to the voice prompt information, the types of voice information including first type voice, second type voice and third type voice;

[0106] A function activation module, used for activating the function of the corresponding voice information according to the voice information when the type of the voice information is the second type of voice;

[0107] The video module is used to generate a second type of video according to the voice information when the type of the voice information is the third type of voice.

[0108] The video module may include:

[0109] A text acquisition unit, used to acquire text corresponding to the voice information;

[0110] A second video generation unit is used to generate a second type of video according to the preset audio corresponding to the text and the third type of voice

[0111] or video modules, which may include:

[0112] A popper acquisition unit, used for generating a dynamic popper corresponding to the frequency of the voice information according to the voice information;

[0113] An avatar acquisition unit, used to acquire the avatar of the user's preset account;

[0114] The third video generation unit is used to generate a second type of video according to the dynamic pop, the avatar and the preset audio corresponding to the third type of voice.

[0115] The voice type determination module may include:

[0116] A first determining unit, configured to determine whether the voice information is a first type of voice or a second type of voice according to a preset language model corresponding to the voice prompt information;

[0117] The second determining unit is used to determine the voice information as a third type of voice when the voice does not belong to the first type of voice and the second type of voice.

[0118] See also Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in the figure is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

[0119] The electronic device includes: a memory and a processor, wherein the processor here may be referred to as a processing device 701 hereinafter, and the memory may include at least one of a read-only memory (ROM) 702, a random access memory (RAM) 703, and a storage device 708 hereinafter, as specifically shown below:

[0120] As shown in the figure, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0121] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 700 with various devices is shown in the figure, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0122] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0123] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0124] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0126] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction; obtains voice information; identifies the semantics corresponding to the voice information based on the voice information; and generates a first type of video based on the semantics.

[0127] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The modules or units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module or unit does not limit the unit itself in some cases. For example, an instruction acquisition module may also be described as a "module for acquiring external information."

[0130] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] According to one or more embodiments of the present disclosure, a method for generating a video using speech is provided, comprising:

[0133] Get a voice function activation command to activate the voice function corresponding to the voice function activation command;

[0134] Get voice information;

[0135] Identify the semantics corresponding to the speech information according to the speech information;

[0136] Generate the first type of video according to semantics.

[0137] According to one or more embodiments of the present disclosure, generating a first type of video according to semantics includes:

[0138] Determine the video dimensions of the generated video according to the semantics, where the video dimensions include at least two of time, place, person, object, scenic spot, emotion, speech-to-text effect, audio, scene, and special effect;

[0139] Get video material according to video dimensions;

[0140] A first type of video is generated according to video material and semantics.

[0141] According to one or more embodiments of the present disclosure, after obtaining the voice function activation instruction, the method further includes:

[0142] Display voice prompt information, the voice prompt information is used to prompt the user to input voice information according to the voice prompt information.

[0143] According to one or more embodiments of the present disclosure, the voice prompt information includes first type information, second type information and third type information; when the type of the voice information is the first type of voice, the semantics corresponding to the voice information is identified based on the voice information.

[0144] According to one or more embodiments of the present disclosure, after acquiring voice information, the method for generating video from voice further includes:

[0145] Determine the type of voice information according to a preset language model corresponding to the voice prompt information, where the types of voice information include first type voice, second type voice and third type voice;

[0146] When the type of the voice information is the second type of voice, the function corresponding to the voice information is enabled according to the voice information;

[0147] When the type of the voice information is the third type of voice, a second type of video is generated according to the voice information.

[0148] According to one or more embodiments of the present disclosure, generating a second type of video according to voice information includes:

[0149] Get the text corresponding to the voice information;

[0150] Generate a second type of video according to the preset audio corresponding to the text and the third type of voice; or

[0151] Generating a second type of video according to the voice information includes:

[0152] Generate a dynamic popper corresponding to the frequency of the voice information according to the voice information;

[0153] Get the user's preset account avatar;

[0154] The second type of video is generated according to the preset audio corresponding to the dynamic pop, the avatar and the third type of voice.

[0155] According to one or more embodiments of the present disclosure, determining the type of voice information according to a preset language model corresponding to the voice prompt information includes:

[0156] Determining whether the voice information is a first type of voice or a second type of voice according to a preset language model corresponding to the voice prompt information;

[0157] When the voice does not belong to the first type of voice or the second type of voice, the voice information is determined to be the third type of voice.

[0158] According to one or more embodiments of the present disclosure, a device for generating video from speech is provided, including:

[0159] An instruction acquisition module is used to acquire a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction;

[0160] A voice acquisition module is used to acquire voice information;

[0161] A speech recognition module, used to recognize the semantics corresponding to the speech information according to the speech information;

[0162] The video generation module is used to generate a first type of video according to semantics.

[0163] According to one or more embodiments of the present disclosure, the video generation module may include:

[0164] A dimension acquisition unit, used to determine the video dimension of the generated video according to the semantics, where the video dimension includes at least two of time, place, person, object, scenic spot, emotion, speech-to-text effect, audio, scene, and special effect;

[0165] A material acquisition unit, used to acquire video material according to video dimensions;

[0166] The first video generating unit is used to generate a first type of video according to video material and semantics.

[0167] According to one or more embodiments of the present disclosure, the apparatus for generating video from speech may further include:

[0168] The prompt display module is used to display voice prompt information, and the voice prompt information is used to prompt the user to input voice information according to the voice prompt information.

[0169] According to one or more embodiments of the present disclosure, the speech recognition module is specifically used to recognize semantics corresponding to the speech information according to the speech information when the type of the speech information is the first type of speech.

[0170] According to one or more embodiments of the present disclosure, the apparatus for generating video by voice may further include:

[0171] A voice type determination module, used to determine the type of voice information according to a preset language model corresponding to the voice prompt information, the types of voice information including first type voice, second type voice and third type voice;

[0172] A function activation module, used for activating the function of the corresponding voice information according to the voice information when the type of the voice information is the second type of voice;

[0173] The video module is used to generate a second type of video according to the voice information when the type of the voice information is the third type of voice.

[0174] According to one or more embodiments of the present disclosure, a video module may include:

[0175] A text acquisition unit, used to acquire text corresponding to the voice information;

[0176] A second video generation unit is used to generate a second type of video according to the preset audio corresponding to the text and the third type of voice

[0177] or video modules, which may include:

[0178] A popper acquisition unit, used for generating a dynamic popper corresponding to the frequency of the voice information according to the voice information;

[0179] An avatar acquisition unit, used to acquire the avatar of the user's preset account;

[0180] The third video generation unit is used to generate a second type of video according to the dynamic pop, the avatar and the preset audio corresponding to the third type of voice.

[0181] According to one or more embodiments of the present disclosure, the voice type determination module may include:

[0182] A first determining unit, configured to determine whether the voice information is a first type of voice or a second type of voice according to a preset language model corresponding to the voice prompt information;

[0183] The second determining unit is used to determine the voice information as a third type of voice when the voice does not belong to the first type of voice and the second type of voice.

[0184] According to one or more embodiments of the present disclosure, there is provided an electronic device, comprising:

[0185] one or more processors;

[0186] Memory;

[0187] One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more programs are configured to: execute the method of generating video from speech according to any of the above embodiments.

[0188] According to one or more embodiments of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for generating video from speech of any of the above embodiments is implemented.

[0189] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0190] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0191] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for generating video from speech, characterized in that: include: Obtain a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction; Get voice information; Identify semantics corresponding to the voice information according to the voice information; A first type of video is generated according to the semantics; the first type of video is generated according to video material and the semantics; the video material is obtained according to video dimensions; the video dimensions are determined and generated according to the semantics.

2. The method for generating video from speech according to claim 1, characterized in that: The video dimensions include at least two of time, place, person, object, scenic spot, emotion, speech-to-text effect, audio, scene, and special effect.

3. The method for generating video from speech according to claim 1, characterized in that: After obtaining the voice function activation instruction, the method further includes: Display voice prompt information, wherein the voice prompt information is used to prompt the user to input voice information according to the voice prompt information.

4. The method for generating video from speech according to claim 3, characterized in that: The voice prompt information includes first type information, second type information and third type information; the types of the voice information include first type voice, second type voice and third type voice; identifying the semantics corresponding to the voice information according to the voice information; Generating a first type of video according to the semantics includes: When the type of the voice information is the first type of voice, identifying semantics corresponding to the voice information according to the voice information; and generating a first type of video according to the semantics; After acquiring the voice information, the method further includes: Determining the type of the voice information according to a preset language model corresponding to the voice prompt information; When the type of the voice information is the second type of voice, enabling a function corresponding to the voice information according to the voice information; When the type of the voice information is the third type of voice, a second type of video is generated according to the voice information.

5. The method for generating video from speech according to claim 4, characterized in that: The generating a second type of video according to the voice information includes: Obtaining text corresponding to the voice information; generating a second type of video according to the preset audio corresponding to the text and the third type of voice; or The generating a second type of video according to the voice information includes: Generate a dynamic wave corresponding to the frequency of the voice information according to the voice information; Get the user's preset account avatar; A second type of video is generated according to the preset audio corresponding to the dynamic pop, the avatar and the third type of voice.

6. The method for generating video from speech according to claim 4, characterized in that: The determining the type of the voice information according to a preset language model corresponding to the voice prompt information includes: Determining whether the voice information is a first type of voice or a second type of voice according to a preset language model corresponding to the voice prompt information; When the voice does not belong to the first type of voice or the second type of voice, the voice information is determined to be a third type of voice.

7. The method for generating video from speech according to claim 1, characterized in that: The voice function activation instruction is triggered by the user pressing a preset virtual control for a preset time period.

8. A device for generating video from speech, characterized in that: include: An instruction acquisition module, used to acquire a voice function activation instruction to activate the voice function corresponding to the voice function activation instruction; A voice acquisition module is used to acquire voice information; A speech recognition module, used for recognizing the semantics corresponding to the speech information according to the speech information; A video generation module is used to generate a first type of video according to the semantics; the first type of video is generated according to video material and the semantics; the video material is obtained according to video dimensions; the video dimensions are determined and generated according to the semantics.

9. An electronic device, characterized in that: It includes: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the method for generating video from speech according to any one of claims 1-7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for generating video from speech as described in any one of claims 1 to 7 is implemented.

Images