Video generation method, device, terminal device and storage medium

Abstract

The present disclosure provides a video generating method, device, equipment and storage medium, the method includes: firstly obtaining current input data of a shooting device, and determining an icon to be rendered according to the current input data; then rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen; and further generating a target video according to the shooting effect screen. The embodiment of the present disclosure determines an icon to be rendered according to current input data of a shooting device, and automatically renders the icon to be rendered on a target shooting screen, thereby efficiently achieving the effect of enriching video screen content.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on international patent application PCT / CN2022 / 115065 filed on August 26, 2022, and claims priority and benefit of Chinese patent application No. 202111074707.3 filed on September 14, 2021. The entire contents of the above patent application are hereby incorporated by reference.

[0002] Technical Field The present disclosure relates to the field of data processing, and in particular to a video generation method, apparatus, terminal device and storage medium. [Background technology]

[0003] With the continuous development of short video technology, people's interest in video creation is increasing, and how to enrich the video creation scene and make the video content more vivid is the goal pursued by video creators.

[0004] Emoji icons, also known as visual emotion symbols, are currently popular emojis online. During the video editing stage, video creators have enriched the content of their video screens by manually adding Emoji icons to the video screen, but manually adding Emoji icons to the video screen is inefficient in enriching video content. Summary of the Invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present disclosure provide a video generation method, apparatus, terminal device, and storage medium that can improve the efficiency of enriching video screen content.

[0006] In a first aspect, the present disclosure provides a method for manufacturing a method of a semiconductor device comprising: acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; determining an icon to render based on the current input data; Rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, the target shooting screen including a preset number of consecutive video frame screens, starting from the current shooting screen of the shooting device; and generating a target video based on the shooting effect screen.

[0007] In a second aspect, the present disclosure provides a method for manufacturing a method of a semiconductor device comprising: an acquisition module for acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; a determination module for determining an icon to be rendered based on the current input data; a rendering module for rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, the target shooting screen including a preset number of consecutive video frame screens, starting from a current shooting screen of the shooting device; a video generation module for generating a target video based on the shooting effect screen.

[0008] In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon instructions that, when executed on a terminal device, cause the terminal device to perform the above video generation method.

[0009] In a fourth aspect, the present disclosure provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above-mentioned video generation method is realized.

[0010] In a fifth aspect, the present disclosure provides a computer program product comprising a computer program / instructions which, when executed by a processor, implements the above video generation method.

[0011] Compared with the prior art, the technical solutions provided by the embodiments of the present disclosure have at least the following advantages:

[0012] An embodiment of the present disclosure provides a video generating method, which first obtains current input data of a shooting device, and determines an icon to be rendered based on the current input data. Then, renders the icon to be rendered on a target shooting screen to obtain a shooting effect screen. Then, generates a target video based on the shooting effect screen. The embodiment of the present disclosure determines the icon to be rendered based on the current input data of a shooting device, and automatically renders the icon to be rendered on the target shooting screen, thereby efficiently achieving the effect of enriching video screen content. [Brief explanation of the drawings]

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0014] In order to more clearly describe the embodiments of the present disclosure or the solutions in the prior art, the drawings that need to be used in describing the embodiments or the prior art are briefly introduced below, and it is obvious that a person skilled in the art can obtain other drawings based on these drawings without any creative efforts.

[0015] [Figure 1] FIG. 1 is a flowchart of a video generation method provided by an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram illustrating an example of a result of rendering an icon to be rendered on a target shooting screen provided by an embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram illustrating an example of a result of rendering an icon to be rendered on a target shooting screen, provided by an embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram illustrating an example of a video editing interface provided by an embodiment of the present disclosure. [Figure 5] FIG. 5 is a diagram illustrating an example of a time setting interface provided by an embodiment of the present disclosure. [Figure 6] FIG. 6 is a diagram illustrating an example of an edit box for a target icon provided by an embodiment of the present disclosure. [Figure 7] FIG. 7 is a diagram illustrating an example of a video recording interface of a camera device provided by an embodiment of the present disclosure. [Figure 8] FIG. 8 is a diagram illustrating an example of a pop-up window displaying permission request information provided by an embodiment of the present disclosure. [Figure 9] FIG. 9 is a diagram illustrating an example of a target video generated by an embodiment of the present disclosure. [Figure 10]FIG. 10 is a schematic structural diagram of a video generating device provided by an embodiment of the present disclosure. [Figure 11] FIG. 11 is a schematic structural diagram of a terminal device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0016] In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure are further described below. It should be noted that, unless there is a contradiction, the embodiments and feature portions of the embodiments of the present disclosure can be combined.

[0017] Although numerous specific details are set forth in the following description in order to provide a thorough understanding of the present disclosure, the present disclosure may be practiced in other ways different from those described herein. Obviously, the embodiments in the specification are only a portion of the embodiments of the present disclosure, but not all of the embodiments.

[0018] Currently, during the video editing stage, video creators can manually add emoji icons to the video screen to enrich the video screen content and make the created video content more vivid, but manually adding emoji icons is inefficient and affects the progress of video creation.

[0019] Therefore, an embodiment of the present disclosure provides a video generation method: first, obtain current input data of a shooting device, and determine an icon to be rendered based on the current input data; then, render the icon to be rendered on a target shooting screen to obtain a shooting effect screen; and then generate a target video based on the shooting effect screen. The embodiment of the present disclosure determines the icon to be rendered based on current input data of a shooting device, and automatically renders the icon to be rendered on the target shooting screen, thereby efficiently achieving the effect of enriching video screen content.

[0020] Based on this, an embodiment of the present disclosure provides a video generation method. Referring to Figure 1, there is shown a flowchart of the video generation method provided by the embodiment of the present disclosure. The method includes the following steps:

[0021] S101: Current input data of the imaging device is obtained.

[0022] The current input data includes audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, with the current input audio frame of the image capture device as the end frame, and the image input data including consecutive video frame screens for a preset number, with the current capture screen of the image capture device as the end frame.

[0023] Both the preset time and the preset number can be set in advance. For example, the preset time can be set to 1 second or 3 seconds, and the preset number can be set to 1 frame or 5 frames.

[0024] In an embodiment of the present disclosure, the audio input data may be background music selected when recording a video through a camera device or words spoken by a user when recording a video, but the present disclosure is not limited thereto. The camera device may be a video recording tool provided by an application program with a video recording function, and the application program may be installed on a terminal device such as a smartphone or a tablet computer.

[0025] For example, assuming that the current input data includes audio input data and the preset time is 3 seconds, if the current background music has not been played until 3 seconds, for example, the current background music has been played until 2 seconds, the acquired audio input data is continuous audio frame data of the background music from 0 seconds to 2 seconds; if the current background music has been played for more than 3 seconds, for example, the current background music has been played until 5 seconds, the acquired audio input data is continuous audio data of the background music from 2 seconds to 5 seconds.

[0026] For example, assuming that the current input data includes image input data, if the preset number is 1, a screenshot of the current shooting screen can be acquired as the image input data, and if the preset number is 3, screenshots of the shooting screen of the previous three frames including the current shooting screen can be acquired as the image input data.

[0027] In an optional embodiment, in a video recording scene of a camera or a preview scene of the shooting screen thereof, audio input data and / or image input data of the camera can be obtained as current input data of the camera.

[0028] Generally, when a user records a video, he or she needs to start a camera to record the video. After the user starts the camera, a preview interface is displayed, and a camera button is displayed on the preview interface. The user clicks the camera button to start recording the video. The preview interface is displayed continuously before the user clicks the camera button. In an embodiment of the present disclosure, before starting recording, audio input data and / or image input data in the preview scene on the camera screen can be acquired as current input data of the camera. After the user clicks the camera button to start recording, audio input data and / or image input data during video recording can be continuously acquired as current input data of the camera.

[0029] S102: Based on the current input data, an icon to be rendered is determined.

[0030] In an embodiment of the present disclosure, a prediction model for rendering an icon can be trained in advance, and the acquired current input data is input into the prediction model, and the prediction model outputs the corresponding icon to be rendered.

[0031] It should be noted that the corresponding prediction models for the audio input data and the image input data can be trained separately to predict the rendering icon, or the prediction models can be trained jointly to predict the rendering icon for the audio input data and the image input data.

[0032] In the embodiments of the present disclosure, the icons to be rendered are not limited to Emoji icons and can be extended to other icons, and the present disclosure is not limited thereto. Emoji icons include various types of icons, such as smiley and emotion icons, human and body icons, skin color and hairstyle icons, animal and nature icons, food and drink icons, travel and place icons, activity icons, item icons, symbol icons, and flag icons. The smile and emotion icons include smiley faces, expressive faces, tongue-out faces, faces with hands, sleeping faces, sick faces, worried faces, negative faces, cat faces, monkey faces, emotion icons, etc.; the human and body icons include open palm icons, single finger icons, gesture icons, body icons, character icons, character gesture icons, character role icons, fictional character icons, character activity icons, character movement icons, character rest icons, family icons, etc.; the skin color and hairstyle icons include skin color icons and hairstyle icons; the animal and nature icons include mammal icons, bird icons, reptile icons, marine animal icons, insect icons, flower icons, and other plant icons; the food and drink icons include fruit and vegetable icons, cooked food icons, beverage icons, tableware icons, etc.; the travel and place icons include building icons, land transportation icons, water transportation icons, etc.; the activity icons include sport icons, game icons, etc.; the item icons include clothing icons, musical instrument icons, office icons, etc.; the symbol icons include constellation icons, function identification icons, etc.

[0033] For example, if audio input data of "I'm going skiing tomorrow" is acquired, it can be determined based on the audio input data that the icon to be rendered is a skiing icon.

[0034] For example, assuming that image input data is acquired in which a person on the captured screen is sticking out their tongue, it can be determined based on the image input data that the icon to be rendered is a tongue-out icon.

[0035] S103: Render the icon to be rendered on the target shooting screen to obtain a shooting effect screen.

[0036] The target shooting screen includes a preset number of consecutive video frame screens, starting from the current shooting screen of the shooting device.

[0037] The preset number can be set in advance, for example, the preset number can be set to 3 frames, 5 frames, etc.

[0038] In the embodiment of the present disclosure, when rendering the icon to be rendered on the target shooting screen, various rendering methods can be used.

[0039] For example, the icon to be rendered is attached to the target shooting screen in the form of a sticker, and in the shooting effect screen obtained by this rendering method, the target shooting screen and the icon to be rendered are independent of each other.

[0040] For example, the icon to be rendered and the target shooting screen can be rendered as a whole, and in the shooting effect screen obtained by this rendering method, the target shooting screen and the icon to be rendered are combined into one.

[0041] S104: Generate a target video based on the shooting effect screen.

[0042] In an embodiment of the present disclosure, the above steps are used to determine relevant icons to be rendered throughout the entire video recording process, and the icons to be rendered are rendered on a target shooting screen to obtain a shooting effect screen. A target video is then generated based on the shooting effect image obtained in each rendering. The target video thus generated includes not only the shooting screen, but also recommended rendering icons according to the shooting screen and the audio output during video recording, thereby enriching the content of the video screen. Appropriate icons are automatically recommended during video recording, eliminating the need for users to manually add icons when editing videos. This increases the enjoyment of video production, while also improving the efficiency of adding icons and saving video production costs.

[0043] According to a video generating method according to an embodiment of the present disclosure, first, current input data of a shooting device is obtained, and an icon to be rendered is determined based on the current input data. Then, the icon to be rendered is rendered on a target shooting screen to obtain a shooting effect screen. Then, a target video is generated based on the shooting effect screen. The embodiment of the present disclosure determines the icon to be rendered based on the current input data of a shooting device, and automatically renders the icon to be rendered on the target shooting screen, thereby efficiently achieving the effect of enriching video screen content.

[0044] In an alternative embodiment, the current input data includes the audio input data, and determining an icon to be rendered based on the current input data comprises: performing speech recognition on the audio input data to obtain a speech recognition result; inputting the speech recognition result into a first recommendation model, processing it through the first recommendation model, and outputting an icon corresponding to the audio input data; determining an icon to be rendered based on the icon corresponding to the audio input data.

[0045] The first recommended model may be obtained through pre-training, by collecting a large number of voice recognition texts and marking rendering icons as training samples, and then training an initial neural network model to obtain a trained first recommended model, which predicts matching icons based on the voice recognition results.

[0046] In an embodiment of the present disclosure, for the acquired audio input data, a conventional voice recognition algorithm is used to perform voice recognition on the audio input data to obtain a voice recognition result, and then the voice recognition result is input to a trained first recommendation model for processing, and an icon corresponding to the audio input data is output from the first recommendation model, and the icon is determined as the icon to be rendered.

[0047] In an alternative embodiment, the current input data includes the image input data, and determining an icon to be rendered based on the current input data comprises: extracting facial expression features and / or body movement features from the image input data; inputting the facial expression features and / or body movement features into a second recommendation model, processing the facial expression features and / or body movement features through the second recommendation model, and then outputting an icon corresponding to the image input data; determining an icon to be rendered based on the icon corresponding to the image input data.

[0048] The second recommendation model may be obtained by pre-training, in which a large number of facial expression features and / or body movement features are collected, rendering icons are marked as training samples, and then an initial neural network model is trained to obtain a trained second recommendation model, which predicts matching icons based on the facial expression features and / or body movement features.

[0049] In an embodiment of the present disclosure, for acquired image input data, an associated facial expression extraction algorithm can be used to extract facial expression features in the image input data, and / or an associated body movement identification algorithm can be used to extract body movement features in the image input data. The extracted facial expression features and / or body movement features can then be input to and processed by a trained second recommendation model, and an icon matching the image input data can be output by the second recommendation model, which can then be determined as the icon to be rendered.

[0050] Optionally, taking into consideration that the icons in the icon library set for the application may change over time, for example, the same icon may become more beautiful, or the range of motion of the same action icon may change, causing the icon output from the first recommended model and / or the second recommended model to no longer completely match the icon in the icon library. In this case, if the icon output from the first recommended model and / or the second recommended model is used as the icon to be rendered, an icon that satisfies the user cannot be obtained. In this case, the icon most similar to the icon output by the first recommended model and / or the second recommended model can be matched from the icon library as the icon to be rendered, and the icon rendered on the target shooting screen can be matched with the icon in the icon library, thereby improving the user's satisfaction with the rendered icon.

[0051] The first and second recommended models in the embodiments of the present disclosure may be recommended models that are locally installed on a terminal device together with an application installation package, or may be recommended models that are stored on a server corresponding to the application, but the present disclosure is not limited thereto.

[0052] In an alternative embodiment, the current input data includes the audio input data and the image input data, and determining an icon to be rendered based on the current input data comprises: determining icons corresponding to said audio input data and said image input data, respectively; and performing a duplicate elimination process on the icons corresponding to the audio input data and the image input data, respectively, to obtain an icon to be rendered.

[0053] When recording a video, a user may sometimes shout "Go for it, go for it" while refueling, and verbally explain the action. In this case, the same icon can be obtained based on the acquired audio input data and image input data. However, when both the icon corresponding to the audio input data and the image corresponding to the image input data are rendered on the target shooting screen, two identical icons will be displayed on the target shooting screen, resulting in overlapping icons and affecting the user's visual experience. In response to this phenomenon, in an embodiment of the present disclosure, the overlapping icons can be deleted from the icons corresponding to the determined audio input data and image input data, and only one icon can be rendered, and the overlapping icons can be deleted, thereby avoiding the rendering of the same icons on the target shooting screen and simplifying the video screen.

[0054] In an alternative embodiment, before rendering the icon to be rendered on the target shooting screen, rendering parameter values ​​corresponding to the icon to be rendered may be determined first, where the rendering parameter values ​​include a rendering position coordinate, a rendering icon size, a rendering angle, and / or a rendering time. Correspondingly, rendering the icon to be rendered on the target shooting screen to obtain a shooting effect screen includes rendering the icon to be rendered on the target shooting screen based on the rendering parameter values ​​to obtain a shooting effect screen.

[0055] For example, different rendering position coordinates may be set according to different current input data. Specifically, if the current input data is audio input data, the rendering position coordinates may be any position coordinates other than the face area on the target shooting screen, such as a blank area such as the upper left or upper right corner of the target shooting screen. A face identification algorithm is used to identify faces in the shooting screen and determine a face area range, so that the face area range is avoided when determining the rendering position coordinates, and the rendered icon does not cover the face in the screen. If the current input data is image input data, the rendering position coordinates may be determined by the position coordinates of facial expression features and / or body movement features in the image input data, and the rendering position coordinates may be close to the position coordinates of the facial expression features and / or body movement features. That is, if the current input data is image input data, the icon to be rendered is rendered at a position as close as possible to the corresponding facial expression and / or body movement.

[0056] For example, the size, angle, and time of the rendering icon may be preset, and may be set by the system or may be set by the user, and the present disclosure is not limited thereto.

[0057] In an embodiment of the present disclosure, based on the rendering parameter values ​​corresponding to the determined icon to be rendered, the icon to be rendered can be rendered on the target shooting screen to obtain a shooting effect screen.

[0058] 2 is a diagram illustrating an example of the result of rendering an icon to be rendered on a target shooting screen provided by an embodiment of the present disclosure. FIG. 2 shows a frame of the shooting effect screen obtained by rendering. As shown in FIG. 2, when recording a video, a user may say, "I'm going skiing tomorrow," and the icon to be rendered can be matched with "skiing" according to the audio data. The icon is then rendered on the video screen according to the rendering position coordinates, the size of the rendering icon, the rendering angle, and the rendering time, as shown by reference numeral 21 in FIG. 2.

[0059] FIG. 3 is a diagram illustrating an example of the result of rendering an icon to be rendered on a target shooting screen provided by an embodiment of the present disclosure. FIG. 3 shows a frame of a shooting effect screen obtained by rendering. As shown in FIG. 3, when a user covers their eyes during video recording, the icon to be rendered is matched with "covering eyes" according to the image data, and the rendering position coordinates of the icon are determined based on the position of the user's covering eyes in the image. For example, the rendering position coordinate is determined as the left side of the face, and the icon is rendered on the video screen according to the rendering position coordinate, the size of the rendering icon, the rendering angle, and the rendering time, as shown by reference numeral 31 in FIG. 3.

[0060] In an optional embodiment, for the generated target video, the target video can be played in a video editing interface, and in response to a selection operation of a target icon on the playback screen of the target video, an editing track corresponding to the target icon is displayed, and the rendering time of the target icon is adjusted based on the editing track.

[0061] FIG. 4 is a diagram illustrating an example of a video editing interface provided by an embodiment of the present disclosure. As shown in FIG. 4, a target video is being played in the video editing interface, and an eye-covering icon, such as the icon 41 in FIG. 4, is displayed on the video. When a user clicks on the icon 41, a corresponding editing track is displayed near the icon, and the editing track is represented by the icon 42 in FIG. 4. As can be seen from FIG. 4, the editing track includes text-to-speech, time setting, and editing. Here, clicking on text-to-speech reads out the audio input data corresponding to the icon. The time setting allows the duration of the icon to be set. The editing allows the position, size, and angle of the icon to be adjusted, and the icon can also be deleted. When a user clicks on time setting, the user enters the time setting interface shown in FIG. 5, where the user can set the duration of the icon 41.

[0062] As shown in Fig. 5, the playback of the target video is paused in the time setting interface. A sticker time setting window is displayed in Fig. 5, and by adjusting the sticker time setting window as shown by reference numeral 51, the user can set the duration of the icon shown by reference numeral 41. As shown in Fig. 5, the sticker time setting window is located between 0.3 seconds (in Fig. 5, seconds are represented by s) and 1.3 seconds of the target video, that is, the duration of the icon shown by reference numeral 41 is set as 1 second, and the time setting interface displays the information "sticker duration is selected as 1.0 s".

[0063] In an embodiment of the present disclosure, a target video is played in a video editing interface, and according to the selection operation of a target icon on the playback screen of the target video, an editing track corresponding to the target icon is displayed, and the rendering time of the target icon is adjusted based on the editing track, so that the user can set the display time of the icon as needed and flexibly adjust the display time of the icon.

[0064] In an optional embodiment, when playing a target video in the video editing interface, the user can further perform a preset operation on a selected target icon, thereby adjusting the rendering position coordinates, the size and / or the rendering angle of the target icon according to the preset operation on the target icon on the playback screen.

[0065] The preset operation may be a zoom operation, a rotation operation, and a drag operation on the target icon, where the zoom operation can adjust the size of the rendering icon of the target icon, the rotation operation can adjust the rendering angle of the target icon, and the drag operation can adjust the rendering position coordinate of the target icon.

[0066] For example, a user can click on Edit in the editing track indicated by reference numeral 42 in Fig. 4, after which an edit box will appear around the target icon indicated by reference numeral 41. As shown in Fig. 6, the box indicated by reference numeral 61 is the edit box of the target icon. By performing a zoom operation on any side of the edit box indicated by reference numeral 61, the size of the target icon indicated by reference numeral 41 can be adjusted. By performing a rotation operation on the edit box indicated by reference numeral 61, the angle of the target icon indicated by reference numeral 41 can be adjusted. By performing a drag operation on the edit box indicated by reference numeral 61, the position of the target icon indicated by reference numeral 41 can be adjusted. Also, as shown in Fig. 6, a delete symbol "X" is further displayed in the upper right corner of the edit box. By clicking on the delete symbol, the target icon indicated by reference numeral 41 can be deleted.

[0067] In the embodiment of the present disclosure, the rendering position coordinates, the size and / or the rendering angle of the target icon are adjusted according to the preset operation on the target icon on the playback screen, thereby realizing the flexible adjustment of the rendering icon and enhancing the enjoyment of video creation.

[0068] As people pay more and more attention to the security of personal information, users usually want to protect their information in all aspects, and the automatic recommendation of rendering icons needs to obtain audio input data and image input data when recording a user's video. In order to inform the user of this situation, in an optional embodiment of the present disclosure, before obtaining the current input data of the photographing device, user authorization information for the current input data of the photographing device can be obtained first, and only after authorization is obtained can the icon recommendation function be turned on, and the current input data is obtained to determine the icon to be rendered.

[0069] Illustratively, Fig. 7 is a diagram illustrating an example of a video recording interface of a photographing device provided by an embodiment of the present disclosure. As shown in Fig. 7, an icon toolbar indicated by reference numeral 71 in Fig. 7 is displayed in the video recording interface. When a user clicks on the icon toolbar, permission request information is displayed in the form of a pop-up window. As shown in Fig. 8, the permission request information is used to request permission to obtain data such as the user's audio, movements, and facial expressions. When the user selects permission, an icon recommendation function indicated by reference numeral 72 in Fig. 7 is turned on. After the icon recommendation function is turned on, the current input data of the photographing device can be obtained to recommend an icon.

[0070] For example, the icon recommendation function indicated by the reference numeral 72 is in the off state (circle on the left) by default. When the user clicks on the icon recommendation function, a pop-up window will appear as shown in FIG. 8, requesting permission to obtain data such as the user's audio, movements, and facial expressions. If the user selects permission, the icon recommendation function will switch from the off state to the on state (circle on the right), and then the current input data of the imaging device can be obtained to recommend an icon.

[0071] FIG. 9 illustrates an example of a target video generated by an embodiment of the present disclosure. For example, if a user utters the phrase, "It's a beautiful day today. I went to the park, and then I got a sudden phone call asking me to come in to work overtime," during video recording, it is determined that the icons to be rendered include a sun icon based on "It's a beautiful day today," and the sun icon is displayed on the subsequent video screen, as indicated by reference numeral 91. The video screen displaying the sun icon corresponds to the audio data "I went to the park." Based on the audio data, it is determined that the icons to be rendered are a park icon, and the park icon is displayed on the subsequent video screen, as indicated by reference numeral 92. According to the audio data "I got a sudden phone call asking me to come in to work overtime," it is determined that the icons to be rendered are a telephone icon and a work icon, and the telephone icon and the work icon are displayed on the video screen, as indicated by reference numerals 93 and 94. It can be seen that since it takes some time to match the icons to be rendered and to render the icons on the recording screen, there is a certain delay in the display of the icons compared to the audio data.

[0072] Based on the above-mentioned method embodiment, the present disclosure further provides a video generation device. Referring to Figure 10, a schematic structural diagram of a video generation device provided by an embodiment of the present disclosure is shown. The video generation device 100 includes: an acquisition module 110 for acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; a determination module 120 for determining an icon to render based on the current input data; a rendering module 130 for rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, the target shooting screen including a preset number of consecutive video frame screens, starting from a current shooting screen of the shooting device; and a video generating module 140 for generating a target video based on the shooting effect screen.

[0073] In an optional embodiment, the acquisition module 110 is used to acquire audio input data and / or image input data of the photographing device as current input data of the photographing device in a video recording scene of the photographing device or a preview scene of its shooting screen.

[0074] In an alternative embodiment, the current input data includes the audio input data, and the decision module 120 specifically: performing speech recognition on the audio input data to obtain a speech recognition result; inputting the speech recognition result into a first recommendation model, processing the result through the first recommendation model, and then outputting an icon corresponding to the audio input data; It is used to determine the icon to be rendered based on the icon corresponding to the audio input data.

[0075] In an alternative embodiment, the current input data includes the image input data, and the determination module 120 specifically: extracting facial expression features and / or body movement features from the image input data; inputting the facial expression features and / or body movement features into a second recommendation model, processing the facial expression features and / or body movement features through the second recommendation model, and then outputting an icon corresponding to the image input data; It is used to determine an icon to be rendered based on the icon corresponding to the image input data.

[0076] In an alternative embodiment, the current input data includes the audio input data and the image input data, and the decision module 120 specifically: determining icons corresponding to the audio input data and the image input data, respectively; The icon processing unit 100 executes a duplicate elimination process on the icons corresponding to the audio input data and the image input data, respectively, to obtain an icon to be rendered.

[0077] In an alternative embodiment, the video production device 100: The device further includes a rendering parameter determination module for determining rendering parameter values ​​corresponding to the icon to be rendered, where the rendering parameter values ​​include a rendering position coordinate, a rendering icon size, a rendering angle, and / or a rendering time.

[0078] Correspondingly, the rendering module 130 specifically: It is used to render the icon to be rendered on the target shooting screen based on the rendering parameter value, to obtain a screen of shooting effects.

[0079] In an alternative embodiment, the video production device 100: a video playback module for playing the target video in a video editing interface; a display module for displaying an edit track corresponding to a target icon in response to a selection operation of the target icon on a playback screen of the target video; and a time adjusting module for adjusting a rendering time of the target icon based on the editing track.

[0080] In an alternative embodiment, the video production device 100: The display device further includes a parameter adjusting module for adjusting a rendering position coordinate, a size and / or a rendering angle of the target icon according to a preset operation on the target icon on the playback screen.

[0081] In an alternative embodiment, the video production device 100: The camera further includes a permission acquisition module for acquiring user permission information for the current input data of the camera.

[0082] The video generation device provided by the embodiments of the present disclosure can execute the video generation method provided by any of the embodiments of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

[0083] In addition to the above methods and apparatus, embodiments of the present disclosure further provide a computer-readable storage medium on which instructions are stored, which, when executed on a terminal device, cause the terminal device to perform the video generation method described in the embodiments of the present disclosure.

[0084] An embodiment of the present disclosure further provides a computer program product including a computer program / instructions that, when executed by a processor, implements the video generation method described in the embodiment of the present disclosure.

[0085] FIG. 11 is a schematic structural diagram of a terminal device provided by an embodiment of the present disclosure.

[0086] 11, a schematic structural diagram suitable for implementing a terminal device 400 in an embodiment of the present disclosure is shown. The terminal device 400 in an embodiment of the present disclosure includes, but is not limited to, mobile devices such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), and in-vehicle devices (e.g., in-vehicle navigation devices), digital TVs, and fixed devices such as desktop computers. The terminal device shown in FIG. 11 is merely an example and does not limit the functionality and scope of use of the embodiment of the present disclosure.

[0087] 11, the terminal device 400 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 401, and may perform various appropriate operations and processes according to programs stored in a read-only memory (ROM) 402 or programs loaded from a storage device 408 into a random access memory (RAM) 403. The RAM 403 further stores various programs and data necessary for the operation of the terminal device 400. The processing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0088] Generally, the following devices may be connected to the I / O interface 405: input devices 406 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 408 including, for example, a magnetic tape, hard disk, etc.; and communication devices 409. The communication devices 409 may enable the terminal device 400 to communicate and exchange data with other devices wirelessly or via wires. While FIG. 11 illustrates the terminal device 400 having various devices, it should be understood that it is not necessary for the terminal device 400 to implement or possess all of the devices shown. Instead, it may implement or possess more or fewer devices.

[0089] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program embodied in a non-transitory computer-readable medium, the computer program including program code for performing the methods illustrated in the flowcharts. In such embodiments, the computer program may be downloaded or installed from a network via the communication device 409, or may be installed from the storage device 408 or from the ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined above in the video generation method according to embodiments of the present disclosure.

[0090] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program. The program may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal, whether propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals can take many forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium, other than a computer-readable storage medium, that can transmit, propagate, or transmit a program used by or in connection with an instruction execution system, apparatus, or device. Program code embodied in a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wire, optical cable, RF (radio frequency), etc., or any suitable combination thereof.

[0091] In some embodiments, clients and servers may communicate using any now known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), an end-to-end network (e.g., an ad hoc end-to-end network), and any now known or later developed network.

[0092] The computer-readable medium may be included in the terminal device, or may exist independently of the terminal device.

[0093] The computer-readable medium stores one or more programs, and when the one or more programs are executed by the terminal device, the terminal device performs the following operations: obtain current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time period, with the current input audio frame of the photographing device as an end frame, and the image input data including a preset number of consecutive video frame screens, with the current photographing screen of the photographing device as an end frame; determine an icon to be rendered based on the current input data, render the icon to be rendered on a target photographing screen to obtain a photographing effect screen, the target photographing screen including the preset number of consecutive video frame screens, with the current photographing screen of the photographing device as a start frame; and generate a target video based on the photographing effect screen.

[0094] Computer program code for carrying out the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and further including conventional procedural programming languages, such as "C" or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0095] The flowcharts and block diagrams in the figures illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, including one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions shown in the blocks may be executed in an order different from that shown in the figures. For example, two blocks shown one after the other may actually be executed substantially in parallel or in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions.

[0096] The units included in the embodiments of the present disclosure can be realized in software or hardware, and the unit names may not limit the units themselves.

[0097] The functionality described herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, typical types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0098] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program used by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of machine-readable storage media include one or more wire-based electrical connections, laptop computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0099] According to one or more embodiments of the present disclosure, the present disclosure provides a method for manufacturing a semiconductor device, comprising: acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; determining an icon to render based on the current input data; Rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, the target shooting screen including a preset number of consecutive video frame screens, starting from the current shooting screen of the shooting device; and generating a target video based on the shooting effect screen.

[0100] According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, the step of obtaining current input data of the imaging device includes: The method includes a step of obtaining audio input data and / or image input data of the photographing device as current input data of the photographing device in a video recording scene of the photographing device or a preview scene of the photographing screen.

[0101] According to one or more embodiments of the present disclosure, in a video generation method provided by the present disclosure, the current input data includes the audio input data, and the step of determining an icon to be rendered based on the current input data includes: performing speech recognition on the audio input data to obtain a speech recognition result; inputting the speech recognition result into a first recommendation model, processing it through the first recommendation model, and outputting an icon corresponding to the audio input data; determining an icon to be rendered based on the icon corresponding to the audio input data.

[0102] According to one or more embodiments of the present disclosure, in a video generation method provided by the present disclosure, the current input data includes the image input data, and the step of determining an icon to be rendered based on the current input data includes: extracting facial expression features and / or body movement features from the image input data; inputting the facial expression features and / or body movement features into a second recommendation model, processing the facial expression features and / or body movement features through the second recommendation model, and then outputting an icon corresponding to the image input data; determining an icon to be rendered based on the icon corresponding to the image input data.

[0103] According to one or more embodiments of the present disclosure, in a video generation method provided by the present disclosure, the current input data includes the audio input data and the image input data, and the step of determining an icon to be rendered based on the current input data includes: determining icons corresponding to said audio input data and said image input data, respectively; and performing a duplicate elimination process on the icons corresponding to the audio input data and the image input data, respectively, to obtain an icon to be rendered.

[0104] According to one or more embodiments of the present disclosure, in a video generating method provided by the present disclosure, before rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, The method further includes determining rendering parameter values ​​corresponding to the icon to be rendered, wherein the rendering parameter values ​​include a rendering position coordinate, a size of the rendering icon, a rendering angle, and / or a rendering time.

[0105] Correspondingly, the step of rendering the icon to be rendered on the target shooting screen to obtain a shooting effect screen includes: The method includes a step of rendering the icon to be rendered on the target shooting screen based on the rendering parameter value to obtain a shooting effect screen.

[0106] According to one or more embodiments of the present disclosure, in the video generating method provided by the present disclosure, after generating a target video based on the shooting effect screen, playing the target video in a video editing interface; displaying an edit track corresponding to a target icon in response to a selection operation of the target icon on a playback screen of the target video; and adjusting a rendering time of the target icon based on the edit track.

[0107] According to one or more embodiments of the present disclosure, there is provided a video generation method, the method comprising: The method further includes adjusting the rendering position coordinates, the size and / or the rendering angle of the target icon according to a preset operation on the target icon on the playback screen.

[0108] According to one or more embodiments of the present disclosure, there is provided a video generation method comprising the steps of: before acquiring current input data of the image capture device, The method further includes obtaining user authorization information for the current input data of the photographing device.

[0109] According to one or more embodiments of the present disclosure, the present disclosure provides a method for manufacturing a semiconductor device, comprising: an acquisition module for acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; a determination module for determining an icon to be rendered based on the current input data; a rendering module for rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, the target shooting screen including a preset number of consecutive video frame screens, starting from a current shooting screen of the shooting device; a video generation module for generating a target video based on the shooting effect screen.

[0110] According to one or more embodiments of the present disclosure, in the video generating device provided by the present disclosure, the acquisition module is specifically used for acquiring audio input data and / or image input data of the shooting device as current input data of the shooting device in a video recording scene of the shooting device or a preview scene of its shooting screen.

[0111] According to one or more embodiments of the present disclosure, in the video generation device provided by the present disclosure, the current input data includes the audio input data, and the determining module specifically: performing speech recognition on the audio input data to obtain a speech recognition result; inputting the speech recognition result into a first recommendation model, processing the result through the first recommendation model, and then outputting an icon corresponding to the audio input data; It is used to determine the icon to be rendered based on the icon corresponding to the audio input data.

[0112] According to one or more embodiments of the present disclosure, in the video generation device provided by the present disclosure, the current input data includes the image input data, and the determination module specifically: extracting facial expression features and / or body movement features from the image input data; inputting the facial expression features and / or body movement features into a second recommendation model, processing the facial expression features and / or body movement features through the second recommendation model, and then outputting an icon corresponding to the image input data; It is used to determine an icon to be rendered based on the icon corresponding to the image input data.

[0113] According to one or more embodiments of the present disclosure, in the video generation device provided by the present disclosure, the current input data includes the audio input data and the image input data, and the determination module specifically: determining icons corresponding to the audio input data and the image input data, respectively; The icon processing unit 100 executes a duplicate elimination process on the icons corresponding to the audio input data and the image input data, respectively, to obtain an icon to be rendered.

[0114] According to one or more embodiments of the present disclosure, there is provided a video generation device, the device comprising: The device further includes a rendering parameter determination module for determining rendering parameter values ​​corresponding to the icon to be rendered, where the rendering parameter values ​​include a rendering position coordinate, a rendering icon size, a rendering angle, and / or a rendering time.

[0115] Correspondingly, the rendering module specifically: It is used to render the icon to be rendered on the target shooting screen based on the rendering parameter value, to obtain a screen of shooting effects.

[0116] According to one or more embodiments of the present disclosure, a video generation device provided by the present disclosure includes: a video playback module for playing the target video in a video editing interface; a display module for displaying an edit track corresponding to a target icon in response to a selection operation of the target icon on a playback screen of the target video; and a time adjusting module for adjusting a rendering time of the target icon based on the editing track.

[0117] According to one or more embodiments of the present disclosure, there is provided a video generation device, the device comprising: The display device further includes a parameter adjusting module for adjusting a rendering position coordinate, a size and / or a rendering angle of the target icon according to a preset operation on the target icon on the playback screen.

[0118] According to one or more embodiments of the present disclosure, there is provided a video generation device, the device comprising: The camera further includes a permission acquisition module for acquiring user permission information for the current input data of the camera.

[0119] According to one or more embodiments of the present disclosure, the present disclosure provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the terminal device realizes any of the video generation methods provided by the present disclosure.

[0120] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to perform any of the video generation methods provided by the present disclosure.

[0121] It should be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another and do not necessarily require or imply that an actual relationship or sequence exists between those entities or operations. Furthermore, the terms "comprise," "contain," or any other variation thereof are intended to cover a non-exclusive inclusion, whereby a process, method, article, or apparatus that includes a set of elements includes not only those elements but also other elements not expressly listed or elements inherent in the process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element.

[0122] The above description is merely a specific embodiment of the present disclosure, intended to enable those skilled in the art to understand or realize the present disclosure. Various modifications to these examples will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other examples without departing from the spirit or scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the examples described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video generation method comprising: acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; determining an icon to render based on the current input data; Rendering the icon to be rendered on a target shooting screen including a preset number of consecutive video frame screens, starting from the current shooting screen of the shooting device, to obtain a shooting effect screen; generating a target video based on the shooting effect screen; After generating a target video based on the shooting effect screen, the video generating method includes: playing the target video in a video editing interface; displaying an edit track corresponding to a target icon in response to a selection operation of the target icon on a playback screen of the target video; and adjusting a rendering time of the target icon based on the edit track; A method for generating video, wherein the position of the edit track is determined by the position of the target icon.

2. The step of acquiring current input data of the imaging device includes: The video generation method of claim 1, further comprising a step of acquiring audio input data and / or image input data of the camera device as current input data of the camera device in a video recording scene of the camera device or a preview scene of the shooting screen thereof.

3. the current input data includes the audio input data, and determining an icon to be rendered based on the current input data includes: performing speech recognition on the audio input data to obtain a speech recognition result; inputting the speech recognition result into a first recommendation model, processing the result through the first recommendation model, and outputting an icon corresponding to the audio input data; and determining the icon to be rendered based on an icon corresponding to the audio input data.

4. the current input data includes the image input data, and determining an icon to be rendered based on the current input data comprises: extracting facial expression features and / or body movement features from the image input data; inputting the facial expression features and / or body movement features into a second recommendation model, and outputting an icon corresponding to the image input data after processing by the second recommendation model; and determining the icon to be rendered based on an icon corresponding to the image input data.

5. the current input data includes the audio input data and the image input data, and determining an icon to be rendered based on the current input data includes: determining icons corresponding to said audio input data and said image input data, respectively; 2. The video generating method of claim 1, further comprising the step of: performing a duplicate elimination process on icons corresponding to the audio input data and the image input data, respectively, to obtain the icon to be rendered.

6. Before rendering the icon to be rendered on the target shooting screen to obtain a shooting effect screen, the video generating method includes: determining rendering parameter values ​​corresponding to the icon to be rendered, the rendering parameter values ​​including: a rendering position coordinate, a rendering icon size, a rendering angle, and / or a rendering time; Correspondingly, the step of rendering the icon to be rendered on the target shooting screen to obtain a shooting effect screen includes: The video generating method of claim 1 , further comprising: rendering the icon to be rendered on the target shooting screen based on the rendering parameter value to obtain a shooting effect screen.

7. The video generation method comprises: The video generating method according to claim 1, further comprising adjusting the rendering position coordinates, the size and / or the rendering angle of the target icon according to a preset operation on the target icon on the playback screen.

8. Before obtaining the current input data of the imaging device, the video generating method includes: The video generating method of claim 1 , further comprising the step of obtaining user authorization information for the current input data of the imaging device.

9. an acquisition module for acquiring current input data of a photographing device, the current input data including audio input data and / or image input data, the audio input data including consecutive audio frame data for a preset time, the last frame of which is a currently input audio frame of the photographing device, and the image input data including consecutive video frame screens for a preset number, the last frame of which is a currently photographed screen of the photographing device; a determination module for determining an icon to be rendered based on the current input data; a rendering module for rendering the icon to be rendered on a target shooting screen to obtain a shooting effect screen, the target shooting screen including a preset number of consecutive video frame screens, starting from a current shooting screen of the shooting device; a video generation module for generating a target video based on the shooting effect screen, The video generation device a video playback module for playing the target video in a video editing interface; a display module for displaying an edit track corresponding to a target icon in response to a selection operation of the target icon on a playback screen of the target video; a time adjusting module for adjusting a rendering time of the target icon based on the editing track; A video generating device, wherein the position of the edit track is determined by the position of the target icon.

10. 9. A computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to implement the video generation method of any one of claims 1 to 8.

11. 9. A terminal device comprising: a memory; a processor; and a computer program stored in the memory and executable on the processor, the terminal device realizing the video generation method according to any one of claims 1 to 8 when the processor executes the computer program.

Images