Video generation method, device and equipment

Abstract

The embodiment of the invention discloses a video generation method, device and equipment, and the method comprises the steps: determining a video copywriting text, and obtaining a material and an image paster containing an explanation image; segmenting the video copywriting text into clause texts, and converting the clause texts into corresponding voices to form a voice file; driving a mouth-shaped area and a face area in the image sticker according to the clause text to form an explanation image mouth-shaped driving sticker; and generating an explanation video according to the voice file, the material and the explanation image mouth shape driving sticker. According to the method and the device, contents such as voice dubbing and mouth shape driving are conveniently completed, a story explanation video is conveniently generated, a vivid deduction effect of performing text matching explanation by using an explanation image can be obtained, and the video creation difficulty is effectively reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of Internet technology, and specifically to a video generation method, device and equipment. Background Art

[0002] With the development of Internet technology, more and more users like to create videos and share them. Usually, users complete video creation through video production tools or apps. In the process of video creation, users need to write video copy text, configure pictures, videos, audio, etc., and adjust their matching relationships. The existing video creation method requires users to have high professional knowledge and has a high creation threshold, which is not convenient for users with weak professional knowledge to create videos. Summary of the invention

[0003] In view of the above problems, the present application proposes a video generation method, device and equipment to solve the following problems: the existing video creation method requires users to have high professional knowledge, the creation threshold is high, and it is not convenient for users with weak professional knowledge to create videos.

[0004] According to one aspect of an embodiment of the present application, a video generation method is provided, including:

[0005] Determine the video copy text, obtain the material and image stickers containing the narrator image;

[0006] The video copy text is divided into sentence texts, and the sentence texts are converted into corresponding voices to form voice files;

[0007] According to the sentence text, the mouth shape area and the face area in the image sticker are driven to form the commentary image mouth shape driven sticker;

[0008] Generate an explanation video by driving stickers based on voice files, materials and the mouth shape of the explanation image.

[0009] Furthermore, determining the video copy text further includes:

[0010] Obtain the style information selected by the user and the video copy prompt information entered;

[0011] Utilize the large language model in the AI ​​model to generate video copy text based on style information and video copy prompt information.

[0012] Furthermore, after determining the video copy text, the method further includes:

[0013] Detect whether the video copy contains sensitive words;

[0014] If so, filter the sensitive words in the video copy.

[0015] Furthermore, the method further comprises:

[0016] Detect sensitive words in the video copy prompt information entered by the user;

[0017] If the video copy prompt information contains sensitive words, a re-entry prompt is generated for the user to re-enter the video copy prompt information according to the re-entry prompt.

[0018] Furthermore, obtaining the image sticker including the interpretation image further includes:

[0019] Cut out the image material uploaded by the user to obtain image stickers.

[0020] Furthermore, performing image extraction on the image material uploaded by the user to obtain the image sticker further includes:

[0021] The image processing model in the AI ​​model is used to identify the image material, determine the position information of the commentary image in the image material, and extract the image stickers from the image material based on the position information.

[0022] Furthermore, dividing the video copy text into sentence texts, and converting the sentence texts into corresponding voices to form a voice file further includes:

[0023] Get the audio and video information selected by the user;

[0024] Use the large language model in the AI ​​model to segment the video text into sentence text according to the preset text segmentation rules;

[0025] The text-to-speech model in the AI ​​model is used to convert the sentence text into speech corresponding to the video timbre information to form a voice file.

[0026] Furthermore, the mouth shape area and the face area in the image sticker are driven according to the sentence text to form the explanation image mouth shape driven sticker, which further includes:

[0027] The mouth shape driven model in the AI ​​model is used to identify the speech features corresponding to each sentence text, and the speech features are mapped to the facial expressions of the narrator in the image stickers. The mouth shape area and facial area in the image stickers are driven to change with the speech features, thus forming a mouth shape driven sticker of the narrator.

[0028] Furthermore, obtaining the material further includes:

[0029] Obtaining the material selected by the user; and / or,

[0030] Utilize the large language model in the AI ​​model to extract the subject information and / or keywords of the video copy text, and recall materials associated with the subject information and / or keywords from the material library.

[0031] Furthermore, generating the explanation video according to the voice file, the material and the explanation image mouth shape driving sticker further includes:

[0032] Create master track, audio track, subtitle track and sticker track;

[0033] Add the material to the main track, add the voice file to the audio track, add the sentence text as subtitle information to the subtitle track, and add the commentary image mouth shape driven stickers to the sticker track;

[0034] Record the correspondence between the commentator's mouth shape driven stickers, subtitle information and voice files, set the display duration of the commentator's mouth shape driven stickers and the subtitle information to be the same as the playback duration of the corresponding audio in the voice file, and set the start display time of the commentator's mouth shape driven stickers and the start display time of the subtitle information according to the start playback time of the corresponding audio in the voice file.

[0035] According to another aspect of an embodiment of the present application, there is provided a video generating device, including:

[0036] An acquisition module, adapted to determine the video copy text, acquire the material and the image stickers including the narrator image;

[0037] The text segmentation and speech conversion module is suitable for segmenting the video copy text into sentence texts and converting the sentence texts into corresponding speech to form a speech file;

[0038] A driving module, adapted to drive the mouth shape area and the face area in the image sticker according to the sentence text, so as to form the commentary image mouth shape driven sticker;

[0039] The generation module is suitable for generating explanation videos based on voice files, materials and stickers driven by the mouth shape of the explanation image.

[0040] According to another aspect of the embodiment of the present application, there is provided a computing device, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0041] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned video generation method.

[0042] According to another aspect of the embodiments of the present application, a computer storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned video generation method.

[0043] According to another aspect of the embodiments of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned video generation method.

[0044] According to the video generation method, device and equipment provided in the embodiments of the present application, the AI ​​model is introduced into the video creation scene, so that the user can use the artificial intelligence capabilities of the AI ​​model to conveniently complete video text AI creation, voice dubbing, narration image capture, lip shape driving and other contents, conveniently generate story-telling narration videos, and obtain a vivid interpretation effect of using the narration image to explain the text, effectively reducing the difficulty of video creation, and providing users with weak professional knowledge with a video creation method that is easy to operate and has a good experience, effectively improving the user's video creation efficiency and quality.

[0045] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation method of the embodiment of the present application is listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only used for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the embodiments of the present application. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0047] Figure 1 A schematic diagram showing a process of a video generation method according to an embodiment of the present application is shown;

[0048] Figure 2 A schematic diagram showing a flow chart of a video generation method according to another embodiment of the present application is shown;

[0049] Figure 3 A schematic diagram of the interface of a video creation page is shown;

[0050] Figure 4 The schematic diagram of the interface of the video copy prompt information input page is shown;

[0051] Figure 5a The interface diagram of the video copy text AI generation page is shown Figure 1 ;

[0052] Figure 5b The interface diagram of the video copy text AI generation page is shown Figure 2;

[0053] Figure 6 An architectural diagram of video generation is shown;

[0054] Figure 7 An interactive diagram showing a video generation method;

[0055] Figure 8 A timing diagram of a video generation method is shown;

[0056] Fig. 9 A structural block diagram of a video generating device according to an embodiment of the present application is shown;

[0057] Fig.10 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0059] First, the terms involved in one or more embodiments of the present application are explained.

[0060] Video copy text: refers to the text script written for the video content, which is usually used to guide the video production process and provide the oral content in the video.

[0061] Narrative image: refers to the image presented in the video and used to narrate the content of the video. It can be a person, cartoon character, animal, etc.

[0062] AI model (Artificial Intelligence Model): refers to a mathematical model or algorithm trained by machine learning or deep learning in the field of artificial intelligence to solve specific tasks or problems. These models can be divided into different types, covering multiple sub-fields such as supervised learning, unsupervised learning, and reinforcement learning, depending on the technology used and the problem solved.

[0063] TTS (Text-to-Speech): is a technology that converts text into speech. It uses a convolutional neural network to train a model to convert text into audio with a specified tone. TTS can convert written text into spoken language, allowing computers or devices to "read" text content.

[0064] Timeline: refers to the project data obtained by expanding the video in chronological order, including video track, audio track, subtitle track, sticker track, etc.

[0065] Material library: refers to the place used to store and manage various types of materials, which can be pictures, videos, audio, text and other content.

[0066] Large Language Model (LLM): refers to an artificial intelligence language model with large scale, huge amount of training data, and many parameters, such as OpenAI GPT, BERT, etc. This type of model is designed to understand and generate natural language, can better train analysis, reasoning, and output key information of text, and can handle a variety of tasks.

[0067] Material understanding: refers to the technology that uses algorithms to analyze and interpret the content of materials such as pictures and videos.

[0068] Frame extraction: Get picture information from the video at a certain interval.

[0069] Script sorting: refers to matching text with materials so that the generated video can better correspond to the content of the copy.

[0070] Figure 1 A schematic diagram of a video generation method according to an embodiment of the present application is shown. Figure 1 As shown, the method comprises the following steps:

[0071] Step S101, determine the video text, obtain the material and the image sticker containing the narration image.

[0072] In an embodiment of the present application, an AI model is introduced into the video creation scene, allowing users to use the artificial intelligence capabilities of the AI ​​model to conveniently complete video text AI creation, voice dubbing, commentary image capture (also known as matting), lip-shaped driving and other content, effectively reducing the difficulty of video creation.

[0073] When a user wants to create a video, the user can enter the video text created by himself or generated by the AI ​​model in the video creation page of the client corresponding to the video creation. In order to improve the quality of the video and enrich the expressiveness of the video, it is also necessary to obtain the materials required for video creation and image stickers containing narration images. Among them, the materials may include picture materials, video materials, etc.; the image stickers can be static stickers or dynamic stickers, which are not limited here; the narration image in the image sticker can be a person, cartoon character, animal, etc., so as to vividly and realistically narrate the video content in the video through the narration image, effectively improving the fun of the video.

[0074] Step S102, dividing the video copy text into sentence texts, and converting the sentence texts into corresponding voices to form a voice file.

[0075] Among them, the AI ​​model includes intelligent sentence segmentation and TTS functions. Through the intelligent sentence segmentation function, the video copy text is segmented into sentence texts according to the preset text segmentation rules. Specifically, the preset text segmentation rules may stipulate: segmentation with punctuation marks (such as commas, periods, exclamation marks, semicolons, question marks, etc.) as segmentation points, with conjunctions (such as but, therefore, so, etc.) as segmentation points, etc.; in addition, the preset text segmentation rules can also limit the maximum number of characters in a single sentence, for example, the preset text segmentation rules stipulate that the number of characters in a single sentence text obtained by segmentation shall not exceed the preset single sentence maximum character threshold, etc.; after completing the intelligent sentence segmentation, the TTS function is used to convert the segmented sentence texts into corresponding voices one by one, thereby forming a voice file.

[0076] Step S103, driving the mouth shape area and the face area in the image sticker according to the sentence text to form an explanatory image mouth shape driven sticker.

[0077] Specifically, the AI ​​model also includes a mouth shape driving function. Through the mouth shape driving function, the sentence text is associated and matched with the image sticker, and the mouth shape area and facial area in the image sticker are driven according to the sentence text, thereby forming an interpretive image mouth shape driven sticker. The mouth shape, facial expression, etc. of the interpretive image presented in the interpretive image mouth shape driven sticker can change with the change of the voice features corresponding to the sentence text, thereby obtaining a vivid interpretation effect of using the interpretive image to explain the text.

[0078] Step S104, generating an explanation video by driving stickers according to the voice file, the material and the mouth shape of the explanation image.

[0079] After obtaining the voice files, materials and narration image mouth shape driving stickers required for video creation, the narration video can be generated by track laying. The voice files, materials and narration image mouth shape driving stickers can be laid on the timeline and rendered to generate the narration video.

[0080] According to the video generation method provided in the embodiment of the present application, the AI ​​model is introduced into the video creation scene, so that users can use the artificial intelligence capabilities of the AI ​​model to conveniently complete video text AI creation, voice dubbing, lip-shaped driving and other content, conveniently generate story-telling commentary videos, and obtain vivid interpretation effects of using commentary images to match text explanations, effectively reducing the difficulty of video creation, and providing users with weak professional knowledge with a video creation method that is easy to operate and has a good experience, effectively improving the user's video creation efficiency and quality.

[0081] Figure 2 A schematic diagram of a video generation method according to another embodiment of the present application is shown. Figure 2 As shown, the method comprises the following steps:

[0082] Step S201, obtain the style information selected by the user and the video copy prompt information input, and use the large language model in the AI ​​model to generate the video copy text based on the style information and the video copy prompt information.

[0083] In the embodiment of the present application, an AI model is connected, and the AI ​​model can be used to provide functions such as AI creation of video copy text. In order to more accurately generate the video copy text required by the user, several types of style information are pre-set for the user to choose from. The style information is used to characterize the style of the generated video copy text, such as funny, cheerful, warm, serious, sad, etc.

[0084] Users can trigger the copywriting AI creation portal in the client corresponding to video creation to enter the video copywriting prompt information input page. The copywriting AI creation portal can be located in the video creation activity page, video creation page, etc.

[0085] Figure 3 The interface diagram of the video creation page is shown as follows: Figure 3 As shown, the video creation page includes a text box 301 for inputting and displaying video text and a text AI creation portal 302. The user can input the video text he created in the text box 301, or jump to the video text prompt information input page by triggering the text AI creation portal 302, so as to use the AI ​​model to generate the video text.

[0086] Figure 4 The interface diagram of the video copy prompt information input page is shown as follows: Figure 4 As shown in the figure, the video copy prompt information input page includes various style information options, such as Figure 4 The user can select the style information he needs from the funny, cheerful, warm, serious, etc. shown; the video copy prompt information input page also includes a video copy prompt information input box 401 and a generate text control 402, and the user can input the video copy prompt information in the video copy prompt information input box 401, and then submit the style information selected by the user and the input video copy prompt information by triggering the generate text control 402.

[0087] After obtaining the style information selected by the user and the input video copy prompt information, the large language model in the AI ​​model is used to expand the style information and the video copy prompt information, so as to generate a video copy text that conforms to the style information and the video copy prompt information.

[0088] Optionally, the embodiment of the present application also provides a sensitive word detection function, which detects sensitive words in the video copy prompt information input by the user; if the video copy prompt information contains sensitive words, the step of generating the video copy text is not entered, but a re-entry prompt is generated, so that the user can re-enter the video copy prompt information according to the re-entry prompt. The video copy text is not generated until the video copy prompt information input by the user does not contain sensitive words and passes the sensitive word detection.

[0089] Optionally, in order to reduce the occurrence of sensitive words in the video copy text, before returning the generated video copy text to the client for presentation, the video copy text can also be subjected to sensitive word detection to detect whether the video copy text contains sensitive words; if sensitive words are contained, the sensitive words in the video copy text are filtered and then the video copy text is returned; if no sensitive words are contained, the video copy text is returned.

[0090] Figure 5a and Figure 5b The interface diagrams of the video copy text AI generation page are shown respectively Figure 1 And interface diagram Figure 2 , wherein the video copy text AI generation page includes a generation result output box 501, a regeneration control 502 and an application control 503. Figure 5a The figure shows a schematic diagram of the interface in the process of waiting for the large language model to return the video copy text. Figure 5b The figure shows the interface after the large language model returns the video copy text. Figure 5b The "*" in the text is used to represent characters in the video copy. If the user is not satisfied with the video copy generated by the AI ​​model, he can not only trigger the AI ​​model to regenerate a new video copy by triggering the regeneration control 502, but also perform editing operations such as adding, deleting, and modifying the generated video copy in the generation result output box 501; after the user finally determines the video copy required for video creation, he can submit the video copy by triggering the application control 503, thereby returning to Figure 3 In the video creation page shown, the video copy text applied at this time can be presented in the copy box 301.

[0091] Step S202, obtaining materials.

[0092] like Figure 3As shown, the video creation page also includes: a tone selection area 303, a commentary image setting area 304, an upload material area 305 and a one-key generation control 306. The user can enter the material selection page by triggering the + sign in the upload material area 305. The user can select the required material from the material selection page. After the user selects the material to be uploaded, the user returns to the video creation page. The selected uploaded material can be presented in the upload material area 305, such as material 1 in the upload material area 305.

[0093] For obtaining materials, the embodiments of the present application provide multiple ways of obtaining materials. Among them, obtaining materials selected by users; and / or, using the large language model in the AI ​​model to extract the theme information and / or keywords of the video copy text, and recalling materials associated with the theme information and / or keywords from the material library. The materials may include: picture materials, video materials, emoticon stickers, etc.

[0094] If the user does not select or upload a material, the default material provided by the operator can also be used as a backup for video production. In order to match more appropriate, more suitable, and more popular materials for the video copy text, in the embodiment of the present application, materials associated with popular subject information and materials associated with popular keywords are also pre-configured in the material library. Among them, a higher priority can be configured for materials associated with popular subject information and materials associated with popular keywords.

[0095] in, Figure 2 The execution order of obtaining materials in step S202 shown in is only a specific example. Step S202 can be executed before step S206. This application does not specifically limit the execution order of step S202. For example, step S202 can be executed before step S203, or after step S203 or after step S205.

[0096] In order to facilitate the generation of explanation videos, the operator can pre-configure a number of different image stickers for users to choose from. Figure 3 In the video creation page shown, a plurality of image stickers containing the image of the narrator are configured in the narrator image setting area 304, such as image stickers corresponding to image 1 to image 4, from which the user can select the desired image sticker. In addition, the embodiment of the present application also supports using the image material uploaded by the user to determine the image sticker and obtain a personalized image sticker, as shown in step S203.

[0097] Step S203, performing cutout on the image material uploaded by the user to obtain an image sticker.

[0098] Taking into account that the image material may contain other content besides the commentary image, such as background, etc., in the embodiment of the present application, only the commentary image part of the image material can be used as an image sticker in the video to avoid excessive obstruction of the video content by other content in the image material, which helps to improve the visual effect of the video.

[0099] Specifically, the image material is identified by using the image processing model in the AI ​​model, the position information of the commentator image in the image material is determined, and the image sticker is cut out from the image material according to the position information, thereby conveniently realizing the recognition and cutout of the commentator image, wherein the position information may specifically be the minimum circumscribed contour position information of the commentator image in the image material;

[0100] Step S204, dividing the video copy text into sentence texts, and converting the sentence texts into corresponding voices to form a voice file.

[0101] Among them, the AI ​​model can include TTS functions with personalized intelligent sentence segmentation and personalized timbre. Figure 3 The video creation page shown in the figure configures a variety of video timbre information in the timbre selection area 303, such as timbre 1 to timbre 3, from which the user can select the desired video timbre information. Obtain the video timbre information selected by the user; use the large language model in the AI ​​model to segment the video copy text into sentence text according to the preset text segmentation rules; use the text-to-speech model in the AI ​​model to convert the sentence text into speech corresponding to the video timbre information to form a voice file. The voice file contains the audio corresponding to each sentence text. In order to facilitate video production, the correspondence between each sentence text and the audio in the voice file is also recorded.

[0102] Step S205, driving the mouth shape area and the face area in the image sticker according to the sentence text to form an explanatory image mouth shape driven sticker.

[0103] The mouth shape driven model in the AI ​​model is used to identify the speech features corresponding to each sentence text, and the speech features are mapped to the facial expressions of the narrator in the image stickers. The mouth shape area and facial area in the image stickers are driven to change with the speech features, thus forming a mouth shape driven sticker of the narrator.

[0104] Among them, through mouth shape driven model recognition and natural language processing, different mouth shape patterns are understood and associated with specific commands or text to drive the commentary image's mouth shape, facial expressions, etc.

[0105] Step S206, laying out the voice file, the material and the commentary image mouth shape driven stickers on the timeline, and rendering them to generate a commentary video.

[0106] like Figure 3As shown, the user can automatically complete operations such as track laying by triggering the one-key generation control 306, and automatically generate an explanation video. Specifically, create a main track, an audio track, a subtitle track, and a sticker track; add the material to the main track, add the voice file to the audio track, add the sentence text as subtitle information to the subtitle track, and add the explanation image mouth shape driving sticker to the sticker track; in the embodiment of the present application, the explanation image mouth shape driving sticker is formed by driving the mouth area and the facial area in the image sticker according to the sentence text. It can be seen that the explanation image mouth shape driving sticker and the sentence text have a corresponding relationship, that is, the explanation image mouth shape driving sticker and the subtitle information have a corresponding relationship. In order to obtain an explanation video using the explanation image for text matching explanation, it is also necessary to record the corresponding relationship between the explanation image mouth shape driving sticker, the subtitle information, and the voice file. Specifically, the corresponding relationship between the explanation image mouth shape driving sticker and the sentence text and the corresponding relationship between each sentence text and the audio in the voice file can be combined to determine the corresponding relationship between the explanation image mouth shape driving sticker, the subtitle information, and the voice file.

[0107] In addition, the display duration of the commentary image mouth-shaped driven stickers and the display duration of the subtitle information are set to be the same as the playback duration of the corresponding audio in the voice file. The start display time of the commentary image mouth-shaped driven stickers and the start display time of the subtitle information are set according to the start playback time of the corresponding audio in the voice file, thereby conveniently and accurately achieving the synchronous comparison playback of the commentary image mouth-shaped driven stickers, subtitle information and voice files.

[0108] The following is an introduction to the track laying of materials in various scenarios:

[0109] (1) If the user does not upload any material, the default material provided by the operator can be used as a backup for video production.

[0110] (2) If the user uploads an image material or a single video material, the image material or video material can be directly filled into the main track; the image material can be stretched to the same length as the audio track; if the video length is shorter than the audio track, copy and fill it until it can cover the length of the audio track; if the video length is longer than the audio track, directly fill it in completely to retain the original length.

[0111] (3) If the user uploads multiple image materials, the image materials uploaded by the user are filled based on the understanding of the image materials and intelligent matching; the remaining spaces are filled in the order of upload until all the spaces are filled.

[0112] (4) If the user uploads multiple video materials, the video materials uploaded by the user will be filled in based on the understanding of the video materials and intelligent matching; among them, if the sum of the video materials is less than the text reading audio track, they will be copied and filled in order to cover the length of the audio track; if the sum of the video materials is less than the audio track, they will be directly filled in to retain the original length.

[0113] (5) If the multiple materials uploaded by the user include image materials and video materials, the materials uploaded by the user are filled according to the understanding of image materials, video materials and intelligent matching; the remaining vacancies are filled in the order of uploading, and the video materials are truncated when encountering keyword illustrations. The truncated video materials are filled into the empty positions behind. If the duration of the truncated video materials is less than a certain duration (such as 1s), they can be directly discarded. If there are still vacancies after filling one round, they are filled again according to the order of uploading, and the above process is repeated until all the vacancies are filled.

[0114] (6) The video material or picture material associated with the keyword configured by the operation has a higher priority among all types of materials. The sentence text (i.e., single sentence) in which the keyword is recognized is filled with the video material or picture material associated with the keyword configured by the operation.

[0115] (7) The emoticon stickers associated with keywords configured by the operator have a higher priority among all types of materials. For the sentence texts that recognize the keywords and fill in the emoticon stickers associated with keywords configured by the operator, if a single sentence text hits multiple times, the matching emoticon stickers can be finally determined based on the preset maximum value (for example, 5).

[0116] (8) The global sticker material configured by the operation can appear throughout the process.

[0117] In actual applications, for the material track, users can choose not to upload materials or choose not to select the built-in materials configured by the operation. They can use the backup materials configured by the operation through the interface as the backup background of the video. For the sticker track, the sticker position is calculated and issued; among them, the adaptation of the screen display ratio of 16:9 and 9:16 can be prioritized, and the sticker positions corresponding to these two ratios are issued; if the screen display ratio of the device corresponding to the client is not the above two ratios, then it can be selected according to the horizontal and vertical screens. If the screen display ratio is 1:1, the corresponding sticker position can be issued according to the ratio of 16:9. For the video sound information, users are supported to select it by themselves. If the user does not select it, the voice conversion is performed according to the default video sound information. For video background music, there is no video background music by default, and user selection and operation configuration are also supported. In order to facilitate video production, the original sound of the video can be selected as silent. For the commentator image, the commentator image's mouth shape driven stickers are in a carousel by default, and their positions are configured. The commentator image can drive its mouth shape and facial expressions according to the text content of the video copy; the position of the subtitle information is bound to the position of the commentator image by default, and also supports configuration; distinguishing between horizontal and vertical screens, the positions of the commentator image's mouth shape driven stickers and subtitle information are different for horizontal and vertical screens, and their positions also support operational configuration.

[0118] Figure 6 The architecture diagram of video generation is shown in Figure 6As shown, the embodiment of the present application sets up multiple video creation topic entrances in the client. When the user wants to create a video, he can trigger any video creation topic entrance and perform specific video copy text creation, tone selection, narration image setting, material upload and other operations according to the video creation rules; the client provides AI text expression material / style / topic extraction function, label and style information recognition function, text generation function, material text matching video synthesis function, editor fine-tuning function, video upload function, video export function, video publishing function, etc.; the client can be provided with TTS component, text generation component, intelligent sentence component, frame extraction component, upload component, playback component, local tag recognition component and editing component, and provides a variety of video editing engines. The service side is deployed with manuscript and traffic services, traffic distribution recommendation, management background, material middle platform, AI open platform, etc.; among them, the traffic recommendation strategy platform in the manuscript and traffic service interacts with the traffic distribution recommendation to determine the recommendation configuration corresponding to different video creation topic entrances; the style service, label service and video creation rule service in the manuscript and traffic service interact with the management background to determine the style configuration, label configuration, operation announcement activity configuration and video creation rule configuration; the material distribution service in the manuscript and traffic service interacts with the material middle platform to provide materials such as pictures, videos, and animated images; the AI ​​open platform provides material understanding and intelligent matching functions, LLM text generation function, TTS function and lip shape drive function, and performs AI standard service encapsulation.

[0119] Figure 7 An interactive diagram of a video generation method is shown, such as Figure 7 As shown, the user triggers the video creation topic entrance through the client, and the server generates a video creation rule card for the client to display based on the stored video creation rule attributes (such as TTS, video tone information, popular material insertion, etc.); the user edits the script through the client according to the video creation rules, and inputs the script, such as inputting video copy prompt information or video copy text, and can also select the style information of text polishing, or not select the style information, which is not limited here. If the input script is video copy prompt information, the video copy text AI creation is performed, and the server packages and stores the style information, video copy prompt information, keywords, etc., and then calls the service, calls the large language model to expand according to the style information and video copy prompt information, generates a script (such as video copy text) and returns it, wherein sensitive word detection can be performed, and if the video copy text contains sensitive words, the sensitive words therein are filtered and processed before returning. If the input script is video copy text, sensitive word detection can be performed on the input script, and if the video copy text contains sensitive words, the user is prompted to modify it.

[0120] The client outputs and applies the script (such as video copy text), adds and deletes materials, and calls algorithms for material recognition and text matching. The client receives the user's one-click generation control trigger request, and can perform on-site verification of the applied video copy text, materials, image stickers, etc. on the client side. If the verification fails, it will prompt for modification. If the verification passes, it will enter the editor and perform intelligent sentence segmentation on the video copy text to obtain sentence text. The client determines whether the video creation contains materials. If it contains materials, the included materials are used to lay the material main track. If it does not contain materials, the backup materials are used to lay the material main track. The client lays the subtitle track according to the sentence text obtained by intelligent sentence segmentation, and displays the release page after the video generation is completed.

[0121] Figure 8 A timing diagram of a video generation method is shown, wherein: Figure 8 The manuscript server, material middle-end service, AI open platform, review platform and storage system can be deployed on the server side.

[0122] Step S801, the user clicks on the video creation topic entrance displayed in the client.

[0123] Step S802: The client requests the manuscript server to obtain the operation configuration topic.

[0124] Step S803, the manuscript server returns the operation configuration topic.

[0125] Step S804: the user clicks on the free creation area to enter the video creation page.

[0126] Step S805, the client requests the manuscript server to obtain the available style information, video tone information and image stickers.

[0127] Step S806, the manuscript server returns style information, video tone information and image stickers.

[0128] Step S807: the client invokes the system input keyboard.

[0129] Step S808: The user inputs or pastes video copy prompt information.

[0130] Step S809: the user clicks on the selected style information.

[0131] Step S810: The client records the style ID of the style information selected by the user.

[0132] Step S811, the client requests the manuscript server to generate a video copy text corresponding to the style information and the video copy prompt information.

[0133] Step S812: The manuscript server requests the AI ​​open platform to generate a video copy text corresponding to the style information and the video copy prompt information.

[0134] Step S813, the AI ​​open platform returns the generated video copy text.

[0135] Step S814: the manuscript service end requests the review platform to perform sensitive word detection on the video copy text.

[0136] Step S815, the review platform returns whether the video copy text contains sensitive words. If it does not contain sensitive words, the generated video copy text is returned. If it contains sensitive words, the sensitive words in the video copy text are filtered and then returned.

[0137] Step S816: the manuscript server returns the video copy text to the client.

[0138] Step S817, enter the video copy text AI generation page, and refresh the generated result output box to generate the video copy text.

[0139] Step S818: The user clicks Apply or Regenerate.

[0140] Step S819, click on custom style to enter the custom style page.

[0141] Step S820, select a combination style and determine style information.

[0142] Step S821, the client requests the manuscript server to generate a video copy text corresponding to the style information and the video copy prompt information.

[0143] Step S822: The manuscript server requests the AI ​​open platform to generate a video copy text corresponding to the style information and the video copy prompt information.

[0144] Step S823, the AI ​​open platform returns the generated video copy text.

[0145] Step S824, the manuscript service end requests the review platform to perform sensitive word detection on the video copy text.

[0146] Step S825, the review platform returns whether the video copy text contains sensitive words. If it does not contain sensitive words, the generated video copy text is returned. If it contains sensitive words, the sensitive words in the video copy text are filtered and then returned.

[0147] Step S826: the manuscript server returns the video copy text to the client.

[0148] Step S827, the generation result output box of the video copy text AI generation page is refreshed to generate the video copy text.

[0149] Step S828: The user selects video tone information.

[0150] Step S829, the client checks whether the video and audio information is already locally stored. If not, the client downloads the video and audio information.

[0151] Step S830: The client requests the manuscript server to obtain the voice audition TTS file.

[0152] Step S831, the manuscript server requests the material middle-end service to obtain the tone audition TTS file.

[0153] Step S832: the material middle-end service returns the sound audition TTS file to the manuscript server.

[0154] Step S833, the manuscript server returns the tone audition TTS file to the client.

[0155] Step S834: the client plays the timbre audition TTS file.

[0156] Step S835: the user selects image material.

[0157] Step S836, the client records the image material selected by the user, where the image material includes an interpretive image, which may be a static sticker or a dynamic sticker.

[0158] Step S837, the user chooses to add material and enters the material selection page.

[0159] Step S838, the user selects picture material or video material in the album and clicks Next.

[0160] Step S839, the client refreshes the video creation page UI, extracts frames from the selected material, outputs label information, and reads the cached information if relevant information has been cached. Among them, deep learning such as convolutional neural network can be used to extract, train, and match features of the material to detect and identify objects and other contents in the material, and then mark and determine the label information.

[0161] Step S840: The user selects a picture material or a video material in the material library.

[0162] Step S841, the client refreshes the video creation page UI, obtains the tag information corresponding to the material, and records the material information; if the material in the material library has been marked offline, the tag information corresponding to the material can be directly obtained.

[0163] Step S842, the user clicks one-key generation to start the video generation and restoration operation.

[0164] Step S843, the client requests the review platform to perform sensitive word detection on the text in the copy box.

[0165] Step S844, whether the sensitive word detection is passed, if passed, proceed to the next step, if not, a prompt will be given.

[0166] Step S845, the client requests the manuscript server to perform intelligent sentence segmentation and TTS processing on the video copy text, and the request content includes text and tone ID.

[0167] Step S846: The manuscript server requests the AI ​​open platform to perform intelligent sentence segmentation and TTS processing on the video copy text.

[0168] Step S847, the AI ​​open platform returns the sentence text obtained by intelligent sentence segmentation, the voice file processed by TTS and its download address.

[0169] Step S848, the manuscript server returns the sentence text obtained by intelligent sentence segmentation, the voice file processed by TTS and its download address to the client.

[0170] Step S849: the client requests the manuscript server to download the voice file.

[0171] Step S850: The manuscript server requests the AI ​​open platform to download the voice file.

[0172] Step S851, the AI ​​open platform returns the voice file to the manuscript server.

[0173] Step S852: the manuscript server returns the voice file to the client.

[0174] Step S853: The client uploads the material selected by the user to the storage system.

[0175] Step S854: the storage system returns the material file storage address.

[0176] Step S855: The client requests AI intelligent matching, and the request content includes sentence text and material address.

[0177] Step S856: the manuscript service end requests AI intelligent matching from the AI ​​open platform.

[0178] Step S857: The AI ​​open platform downloads materials and performs image recognition processing, performs keyword recognition processing on the sentence text, and performs fuzzy matching of text labels and image labels.

[0179] Step S858: The AI ​​open platform returns the sentence text and material address matching information to the manuscript server.

[0180] Step S859, the manuscript server returns the sentence text and material address matching information to the client.

[0181] Step S860: The client matches the text information and the material information and sorts the scripts.

[0182] Step S861: The client uploads the image material selected by the user to the storage system.

[0183] Step S862: the storage system returns the storage address of the image material.

[0184] Step S863, the client requests the commentary image capture service (i.e., the matting service) from the manuscript server, and the request content includes the storage address of the image material.

[0185] Step S864, the manuscript server requests the AI ​​open platform for the commentary image capture service.

[0186] Step S865, the AI ​​open platform downloads the image material, cuts out the image material to obtain the image sticker, and drives the mouth area and facial area in the image sticker according to the sentence text to form the commentary image mouth shape driven sticker, and uploads the commentary image mouth shape driven sticker to the storage system.

[0187] Step S866, the AI ​​open platform returns the download address of the commentary image mouth shape driven sticker to the manuscript server.

[0188] Step S867, the manuscript server returns the download address of the commentary image mouth shape driven sticker to the client.

[0189] Step S868, the client processes the position information of the commentary image mouth-shaped driving sticker according to the background configuration and matches the corresponding sentence text.

[0190] Step S869, enter the editor to display track laying.

[0191] According to the video generation method provided in the embodiment of the present application, an AI model is introduced into the video creation scene, which provides video production with functions such as material recognition, text sensitive word checking, video copy text AI creation, intelligent segmentation, text material matching, TTS text to speech, narration image capture, lip shape drive, material positioning, etc., and uses voice files, materials and narration image mouth shape drive stickers to lay out the timeline, so as to conveniently obtain an explanation video with subtitles, reading, transitions, background music, filters, etc., and can obtain a vivid interpretation effect of using the narration image for text matching and explanation; this solution effectively reduces the difficulty of video creation, provides users with weak professional knowledge with a video creation method that is easy to operate and has a good experience, effectively improves the user's creation efficiency and quality of video creation, helps to increase the number of effective video creation manuscripts on the video platform, and effectively expands the video resources of the video platform.

[0192] Fig. 9 FIG. 4 shows a structural block diagram of a video generating device according to an embodiment of the present application. Fig. 9 As shown, the device includes: an acquisition module 910, a text segmentation and speech conversion module 920, a driving module 930 and a generation module 940.

[0193] The acquisition module 910 is adapted to: determine the video copy text, acquire the material and the image stickers including the narration image.

[0194] The text segmentation and voice conversion module 920 is adapted to segment the video copy text into sentence texts, and convert the sentence texts into corresponding voices to form a voice file.

[0195] The driving module 930 is adapted to drive the mouth shape area and the face area in the image sticker according to the sentence text to form an explanatory image mouth shape driven sticker.

[0196] The generation module 940 is suitable for: generating an explanation video according to the voice file, the material and the sticker driven by the mouth shape of the explanation image.

[0197] Optionally, the acquisition module 910 is further adapted to: obtain the style information selected by the user and the video copy prompt information input; use the large language model in the AI ​​model to generate the video copy text based on the style information and the video copy prompt information.

[0198] Optionally, the acquisition module 910 is further adapted to: detect whether the video copy text contains sensitive words; if so, filter the sensitive words in the video copy text.

[0199] Optionally, the acquisition module 910 is further adapted to: perform sensitive word detection on the video copy prompt information input by the user; if the video copy prompt information contains sensitive words, generate a re-entry prompt for the user to re-enter the video copy prompt information according to the re-entry prompt.

[0200] Optionally, the acquisition module 910 is further adapted to: cut out the image material uploaded by the user to obtain image stickers.

[0201] Optionally, the acquisition module 910 is further adapted to: utilize the image processing model in the AI ​​model to identify the image material, determine the position information of the commentary image in the image material, and extract the image sticker from the image material based on the position information.

[0202] Optionally, the text segmentation and speech conversion module 920 is further suitable for: obtaining the video timbre information selected by the user; using the large language model in the AI ​​model to segment the video copy text into sentence text according to preset text segmentation rules; using the text-to-speech model in the AI ​​model to convert the sentence text into speech corresponding to the video timbre information to form a voice file.

[0203] Optionally, the driving module 930 is further adapted to: utilizing the mouth shape driving model in the AI ​​model to identify the speech features corresponding to each sentence text, mapping the speech features to the expression of the narrator image in the image sticker, driving the mouth shape area and facial area in the image sticker to change with the speech features, and forming a narrator image mouth shape driven sticker.

[0204] Optionally, the acquisition module 910 is further adapted to: acquire the material selected by the user; and / or, utilize the large language model in the AI ​​model to extract the subject information and / or keywords of the video copy text, and recall the material associated with the subject information and / or keywords from the material library.

[0205] Optionally, generation module 940 is further adapted to: create a main track, an audio track, a subtitle track and a sticker track; add materials to the main track, add voice files to the audio track, add sentence text as subtitle information to the subtitle track, and add commentary image mouth-driven stickers to the sticker track; record the correspondence between commentary image mouth-driven stickers, subtitle information and voice files, set the display duration of the commentary image mouth-driven stickers and the display duration of the subtitle information to be the same as the playback duration of the corresponding audio in the voice file, and set the start display time of the commentary image mouth-driven stickers and the start display time of the subtitle information according to the start playback time of the corresponding audio in the voice file.

[0206] The description of each module above refers to the corresponding description in the method embodiment and will not be repeated here.

[0207] According to the video generation device provided in the embodiment of the present application, an AI model is introduced into the video creation scene, which provides video production with functions such as material recognition, text sensitive word checking, video copy text AI creation, intelligent segmentation, text material matching, TTS text to speech, narration image capture, lip shape drive, material positioning, etc., and uses voice files, materials and narration image mouth shape drive stickers to lay out the timeline, so as to conveniently obtain an narration video with subtitles, reading, transitions, background music, filters, etc., and can obtain a vivid interpretation effect of using the narration image for text matching and explanation; this solution effectively reduces the difficulty of video creation, provides users with weak professional knowledge with a video creation method that is easy to operate and has a good experience, effectively improves the user's video creation efficiency and quality, helps to increase the number of effective video creation manuscripts on the video platform, and effectively expands the video resources of the video platform.

[0208] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the video generation method in any of the above method embodiments.

[0209] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the video generation method in any of the above method embodiments.

[0210] Fig.10 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the computing device.

[0211] like Fig.10 As shown, the computing device may include: a processor (processor) 1002 , a communication interface (Communications Interface) 1004 , a memory (memory) 1006 , and a communication bus 1008 .

[0212] The processor 1002, the communication interface 1004, and the memory 1006 communicate with each other via a communication bus 1008. The communication interface 1004 is used to communicate with other devices such as a client or other server network elements. The processor 1002 is used to execute the program 1010, which can specifically execute the relevant steps in the above-mentioned video generation method embodiment for a computing device.

[0213] Specifically, the program 1010 may include program codes, which include computer operation instructions.

[0214] The processor 1002 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0215] The memory 1006 is used to store the program 1010. The memory 1006 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0216] The program 1010 can be specifically used to enable the processor 1002 to execute the video generation method in any of the above method embodiments. The specific implementation of each step in the program 1010 can refer to the corresponding descriptions in the corresponding steps and units in the above video generation embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiments, which will not be repeated here.

[0217] The algorithm and display provided herein are not inherently related to any specific computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application embodiment described herein, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application embodiment.

[0218] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0219] Similarly, it should be understood that in order to streamline the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed embodiments of the present application require more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the embodiment of the present application.

[0220] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0221] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the embodiments of the present application and to form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0222] The various component embodiments of the embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The embodiments of the present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the embodiments of the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0223] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The embodiments of the present application may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A video generation method, comprising: Determine the video copy text, obtain the material and image stickers containing the narrator image; The video copy text is divided into sentence texts, and the sentence texts are converted into corresponding voices to form a voice file; According to the sentence text, the mouth shape area and the face area in the image sticker are driven to form an explanation image mouth shape driven sticker; The explanation video is generated according to the voice file, the material and the explanation image mouth shape driven sticker.

2. The method according to claim 1, wherein determining the video copy text further comprises: Obtain the style information selected by the user and the video copy prompt information entered; The large language model in the AI ​​model is used to generate the video copy text according to the style information and the video copy prompt information.

3. The method according to claim 1 or 2, after determining the video copy text, the method further comprises: Detecting whether the video text contains sensitive words; If so, the sensitive words in the video copy text are filtered.

4. The method according to claim 2, further comprising: Detect sensitive words in the video copy prompt information entered by the user; If the video text prompt information contains sensitive words, a re-entry prompt is generated, so that the user can re-enter the video text prompt information according to the re-entry prompt.

5. The method according to claim 1, wherein obtaining the image sticker including the interpretive image further comprises: The image material uploaded by the user is cut out to obtain the image sticker.

6. The method according to claim 5, wherein the step of performing image extraction on the image material uploaded by the user to obtain the image sticker further comprises: The image material is identified by using the image processing model in the AI ​​model, the position information of the commentary image in the image material is determined, and the image sticker is cut out from the image material based on the position information.

7. According to the method according to any one of claims 1 to 6, the step of dividing the video copy text into sentence texts and converting the sentence texts into corresponding voices to form a voice file further comprises: Get the audio and video information selected by the user; Using the large language model in the AI ​​model, the video copy text is segmented into sentence texts according to preset text segmentation rules; The text-to-speech model in the AI ​​model is used to convert the sentence text into speech corresponding to the video timbre information to form the speech file.

8. The method according to any one of claims 1 to 7, wherein the step of driving the mouth shape area and the facial area in the image sticker according to the sentence text to form the narration image mouth shape driven sticker further comprises: The mouth shape driven model in the AI ​​model is used to identify the voice features corresponding to each sentence text, and the voice features are mapped to the expression of the narrator image in the image sticker, so that the mouth shape area and facial area in the image sticker are driven to change with the voice features, thereby forming the narrator image mouth shape driven sticker.

9. The method according to any one of claims 1 to 8, wherein obtaining material further comprises: Obtaining the material selected by the user; and / or, The large language model in the AI ​​model is used to extract the subject information and / or keywords of the video copy text, and materials associated with the subject information and / or the keywords are recalled from the material library.

10. The method according to any one of claims 1 to 9, wherein generating the explanation video according to the voice file, the material and the explanation image mouth shape driving sticker further comprises: Create master track, audio track, subtitle track and sticker track; Add the material to the main track, add the voice file to the audio track, add the sentence text as subtitle information to the subtitle track, and add the narration image mouth shape driving sticker to the sticker track; Record the correspondence between the commentary image mouth-shaped driving sticker, the subtitle information and the voice file, set the display duration of the commentary image mouth-shaped driving sticker and the display duration of the subtitle information to be the same as the playback duration of the corresponding audio in the voice file, and set the start display time of the commentary image mouth-shaped driving sticker and the start display time of the subtitle information according to the start playback time of the corresponding audio in the voice file.

11. A video generating device, comprising: An acquisition module, adapted to determine the video copy text, acquire the material and the image stickers including the narrator image; A text segmentation and voice conversion module, adapted to segment the video copy text into sentence texts, and convert the sentence texts into corresponding voices to form a voice file; A driving module, adapted to drive the mouth shape area and the face area in the image sticker according to the sentence text to form an explanation image mouth shape driven sticker; The generation module is suitable for generating an explanation video according to the voice file, the material and the explanation image mouth shape driven sticker.

12. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the video generation method according to any one of claims 1-10.

13. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and wherein the executable instruction enables a processor to execute an operation corresponding to the video generation method according to any one of claims 1 to 10.

14. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the video generation method according to any one of claims 1 to 10.

Images