Video generation method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a video generation method and device, equipment, a medium and a product. The method comprises the following steps: in response to an input operation of a target user for uploading a to-be-processed image, determining a special effect image corresponding to the to-be-processed image; wherein the style types of the special effect image and the to-be-processed image are different; a target special effect video is generated based on the to-be-processed image and the special effect image, and the to-be-processed image and the special effect image are video frames in the target special effect video; and displaying the target special effect video. According to the technical scheme provided by the embodiment of the invention, the effects of processing the received image into the special effect images with different styles and types and generating the special effect video according to the two images with different styles and types are realized.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer processing technology, and in particular, to a video generation method, apparatus, device, medium, and product. Background Art

[0002] Currently, more and more users are using application software to generate corresponding special effects videos.

[0003] When producing special effects, the corresponding special effects videos are mostly generated by relying on the content in the special effects package. However, the content of the existing special effects package is relatively simple, and there are problems such as low content richness of the special effects video and poor picture display effect. Summary of the invention

[0004] The present disclosure provides a video generation method, apparatus, device, medium and product to achieve the effect of processing received images into special effect images of different style types, and generating a special effect video based on two images of different style types.

[0005] In a first aspect, an embodiment of the present disclosure provides a video generation method, the method comprising:

[0006] In response to an input operation of a target user for uploading an image to be processed, determining a special effect image corresponding to the image to be processed; wherein the style types of the special effect image and the image to be processed are different;

[0007] Generate a target special-effect video based on the image to be processed and the special-effect image, wherein the image to be processed and the special-effect image are video frames in the target special-effect video;

[0008] Display the target special effects video.

[0009] In a second aspect, an embodiment of the present disclosure further provides a video generating device, the device comprising:

[0010] A special effect image generation module, configured to determine, in response to an input operation of a target user for uploading an image to be processed, a special effect image corresponding to the image to be processed; wherein the style types of the special effect image and the image to be processed are different;

[0011] A special effect video generation module, used to generate a target special effect video based on the image to be processed and the special effect image, wherein the image to be processed and the special effect image are video frames in the target special effect video;

[0012] The special effects video display module is used to display the target special effects video.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the video generating method as described in any one of the embodiments of the present disclosure.

[0017] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the video generation method as described in any one of the embodiments of the present disclosure.

[0018] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the video generation method as described in any one of the embodiments of the present disclosure.

[0019] The technical solution of the disclosed embodiment determines a special effect image corresponding to the image to be processed in response to the input operation of the target user for uploading the image to be processed; wherein, there is a difference in the style types of the special effect image and the image to be processed, and the effect of the special effect image with different style types is determined through a simple interactive operation, which provides a video generation basis for the subsequent generation of special effect videos; further, based on the image to be processed and the special effect image, a target special effect video is generated, wherein the image to be processed and the special effect image are video frames in the target special effect video; then, the target special effect video is displayed, which solves the problems of low content richness and poor picture display effect of the special effect video in the related art, and realizes the processing of the received image into special effect images with different style types, and generates the effect of the special effect video according to the two images with different style types, thereby enhancing the richness of the special effect video content, improving the special effect display effect of the special effect video, improving the flexibility and intelligence of the special effect video generation method, and improving the special effect application experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 A schematic diagram of a flow chart of a video generation method provided by an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of another video generation method provided by an embodiment of the present disclosure;

[0023] Figure 3 A flowchart of another video generation method provided by an embodiment of the present disclosure;

[0024] Figure 4 A flowchart of another video generation method provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of an effect of a special effect editing page provided in an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of an interface for determining a target style model and a special effect video generation model provided in an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of an interface for determining a target style model and a special effect video generation model provided in an embodiment of the present disclosure;

[0028] Figure 8 A flowchart of a method for determining target effect information provided by an embodiment of the present disclosure;

[0029] Fig. 9 A schematic diagram of a flow chart of a video generation method provided by an embodiment of the present disclosure;

[0030] Fig.10 A schematic diagram of the structure of a video generating device provided by an embodiment of the present disclosure;

[0031] Fig.11 A schematic diagram of the structure of an electronic device provided by the embodiment of the present disclosure for implementing the embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0033] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0034] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0035] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0036] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0037] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0038] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0039] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0040] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0041] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0042] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0043] Before introducing the present technical solution, an exemplary description of the application scenario can be given. The technical solution can be applied to a scenario in which special effects processing is performed on a received image based on pre-configured special effects information to generate a special effects video. For example, after the special effects information is pre-configured, the special effects information can be applied. Furthermore, when an image to be processed is received, the image to be processed can be converted into a special effects image according to the special effects information, and a special effects video including the image to be processed and the special effects image can be generated. When generating a special effects video based on the received image to be processed, the technical solution provided by the embodiment of the present disclosure can be adopted.

[0044] Figure 1 A flow chart of a video generation method provided in an embodiment of the present disclosure is applicable to a scenario in which a special effects video is generated based on a received image. The method can be executed by a video generation device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC, a server, etc.

[0045] like Figure 1 As shown, the method of this embodiment may specifically include:

[0046] S110 . In response to an input operation of a target user for uploading an image to be processed, determining a special effect image corresponding to the image to be processed; wherein the special effect image and the image to be processed have different style types.

[0047] Among them, the target user can be a user who obtains the required special effect video based on the relevant application software. The target user can be any user, optionally including a user registered in the application software, and / or a user who is not registered in the application software and only uses the relevant functions in the application software, etc. The image to be processed can be an image to be processed with special effects. Optionally, the image to be processed can be a default template image, or an image collected based on a camera device set on the terminal device, or an image obtained from a target storage space (such as an image library of the application software, or a local terminal album, etc.) in response to a user's trigger operation, or an image received from an external device and uploaded, or an image extracted from a video, etc. The input operation of uploading the image to be processed can be understood as an operation of uploading the image to be processed to the application software. Optionally, the input operation of uploading the image to be processed includes at least one of the following triggering methods: triggering a preset image upload control; the received audio information includes a trigger word associated with the image upload operation; receiving a preset image upload instruction, etc. In the embodiment of the present disclosure, the image to be processed can be obtained, and further, in the case of receiving an input operation for uploading the image to be processed, the input operation is responded to and the image to be processed is uploaded to the application software. The target video can be any video. The target video can be a video captured by the terminal device; or, it can be a video obtained from a target storage space (such as a video library of an application software, or a local terminal album, etc.) in response to a video selection operation; or, it can be a video uploaded by an external device, etc.

[0048] In an embodiment of the present disclosure, the input operation for uploading an image to be processed may include a variety of implementation methods, optionally including obtaining an uploaded or photographed image as the image to be processed; and / or obtaining a target video and extracting video frames in the target video as the image to be processed.

[0049] As an optional implementation of the first embodiment of the present disclosure, an image upload control can be set in advance in the display interface. When a trigger operation is detected for the image upload control, an upload image path selection page can be popped up, and the selection page can include at least three image upload paths such as local terminal album, image library and third-party upload. Further, when a selection trigger operation is detected for any image upload path, an image display page corresponding to the image upload path can be displayed in the display interface, and the image display page can include multiple images to be uploaded associated with the image upload path, and a selection can be made from the displayed multiple images to be uploaded by selecting the trigger operation. Further, when a selection trigger operation for at least one image to be uploaded is detected, the selected at least one image to be uploaded can be used as an uploaded image. Further, when a selection completion operation is detected (for example, triggering a selection confirmation control or clicking a blank area in the terminal device screen, etc.), it can be determined that an image upload operation is detected. Further, in response to the image upload operation, the uploaded image is used as an image to be processed.

[0050] As another optional implementation of the disclosed embodiment, in the case of receiving an input operation for uploading an image to be processed, in response to the input operation, and based on the video selection operation, any candidate video is selected from the target storage space as the target video. Further, image selection prompt information can be obtained, and based on the image selection prompt information, video frames are extracted from the target video, and the extracted video frames are used as the image to be processed. The image selection prompt information can be used to prompt the video frames expected to be selected in the target video.

[0051] When the image to be processed is obtained, the image to be processed can be subjected to stylized special effects processing to obtain a special effects image corresponding to the image to be processed. Among them, the special effects image can be understood as an image that meets the special effects processing requirements and meets the expected special effects. There are differences in the style types of the special effects image and the image to be processed. In other words, the special effects image can be an image obtained by performing stylized special effects processing on the image to be processed using a style special effect different from the style type of the image to be processed, so that the obtained image can present an image style type that is different from the style type of the image to be processed and consistent with the style type of the style special effect used when performing the stylized special effects processing. The special effects image can be an image of any style type. Optionally, the style type of the special effects image is a comic style, an ink painting style, a cyberpunk style, a sketch style, or the like.

[0052] In the embodiments of the present disclosure, there are at least two ways to determine the special effect image: processing the image to be processed according to the target style obtained by pre-training to obtain the special effect image; or processing the image to be processed according to the stylized processing algorithm to obtain the special effect image.

[0053] It should be noted that when processing an image according to a target style model, different style types may correspond to different target style models. In order to make the final special effect image more compatible with the image to be processed and to make the stylized special effect of the final special effect image better, a corresponding target style model may be determined according to the image to be processed, and the image to be processed may be processed based on the determined target style model.

[0054] Optionally, determining a special effect image corresponding to the image to be processed includes: extracting image features of the image to be processed to determine scene information of the image to be processed based on the image features; and calling a target style model that matches the scene information to process the image to be processed to obtain a special effect image. The advantage of such a setting is that it achieves the effect of determining the scene information of the image based on the image features, and determining and calling a corresponding style model for special effect processing based on the scene information, and achieves the effect of performing scene-based special effect processing on the received image, thereby improving the degree of fit between the special effect image and the image to be processed, and improving the special effect display effect of the special effect image.

[0055] Among them, the image feature can be understood as information that characterizes the key attributes and features in the image. In the embodiment of the present disclosure, in order to improve the degree of adaptation between the special effect image and the image to be processed, feature extraction can be performed on the image to be processed to extract feature information that characterizes the key attributes and features in the image to be processed, and the extracted feature information is used as the image feature of the image to be processed. The image feature can be information based on any form of representation, and optionally, feature information based on vector representation. Scene information can be understood as information that characterizes the overall content and context of the image. Optionally, the scene information can be used to characterize the object type of the target object included in the image to be processed and / or the environment in which the target object is located. Optionally, the object type can include at least one of a user, a pet, a landscape, and a building. The target style model can be a neural network model that takes an image as an input object to stylize the image and outputs an image with a specific stylized special effect. The target style model can be a deep learning model of any model structure. The target style model can be pre-trained and configured in the application software.

[0056] It should be noted that the method of extracting image features of the image to be processed may include at least one, optionally including extracting features of the image to be processed based on a pre-trained feature extraction model; or processing the image to be processed according to a feature extraction algorithm, etc.

[0057] As an optional implementation of the first embodiment of the present disclosure, a variety of scene information can be determined in advance, and target style models corresponding to each scene information can be trained separately. Further, the scene identifier of the scene information can be determined, and the scene identifier can be associated with the target style model and configured in the application software. Further, when the image to be processed is obtained, the image to be processed can be input into the feature extraction model to extract features of the image to be processed based on the feature extraction model, and output image features corresponding to the image to be processed. Further, feature analysis can be performed on the image features to determine the scene information of the image to be processed based on the image features. Further, the target style model adapted to the scene information can be called according to the scene identifier corresponding to the determined scene information, and the image to be processed can be input into the called target style model. Then, the image to be processed is stylized based on the target style model, and a special effect image corresponding to the image to be processed is output. Among them, the style type of the special effect image is different from the style type of the image to be processed, and is consistent with the style type corresponding to the stylized model.

[0058] As another optional implementation of the disclosed embodiment, when the image to be processed is obtained, the image to be processed can be processed according to a preset stylization processing algorithm to obtain a special effect image corresponding to the image to be processed. The style type of the special effect image is different from the style type of the image to be processed, and is consistent with the style type corresponding to the stylization processing algorithm.

[0059] S120. Generate a target special-effect video based on the image to be processed and the special-effect image, wherein the image to be processed and the special-effect image are video frames in the target special-effect video.

[0060] Among them, the target special effects video can be a special effects video for presenting the special effects processing effect of the image to be processed and the special effects image. The image to be processed and the special effects image are video frames in the target special effects video, that is, the video frames included in the target special effects video can include the image to be processed and the special effects image. The image to be processed can be any frame in the target special effects video. The special effects image can also be any frame in the target special effects video. Exemplarily, the image to be processed can be the first video frame in the target special effects video, and the special effects image can be the last video frame in the target special effects video. Furthermore, the video content in the target special effects video can be used to characterize the special effects processing process of transforming the image to be processed into the special effects image. Or, the image to be processed can be the last video frame in the target special effects video, and the special effects image can be the first video frame in the target special effects video. Furthermore, the video content in the target special effects video can be used to characterize the process of transforming the special effects image into the image to be processed.

[0061] It should be noted that, in the target special effects video including the image to be processed and the special effect image, there may be at least two situations: one is that there may be no video frame between the video frame corresponding to the image to be processed and the video frame corresponding to the special effect image, that is, the target special effects video only includes two video frames, namely the image to be processed and the special effect image; the other is that there may be at least one video frame between the video frame corresponding to the image to be processed and the video frame corresponding to the special effect image, and the at least one video frame may be determined based on the image to be processed and the special effect image.

[0062] In the embodiments of the present disclosure, the target special effects video may be generated in at least three ways: splicing the processed image and the special effects image according to a preset video frame splicing order, and using the spliced ​​special effects video as the target special effects video; processing the processed image and the special effects image according to a pre-trained special effects video generation model to generate the target special effects video; or processing the processed image and the special effects image according to a special effects video generation algorithm to generate the target special effects video.

[0063] As an optional implementation of the first embodiment of the present disclosure, when the image to be processed and the special effect image corresponding to the image to be processed are obtained, the image to be processed and the special effect image can be input into a pre-trained special effect video generation model. Furthermore, the image to be processed and the special effect image can be processed based on the special effect video generation model to generate at least one generated image between the image to be processed and the special effect image. Furthermore, a target special effect video with the image to be processed as the first frame and the special effect image as the last video frame can be generated based on the image to be processed, the special effect image and at least one generated image.

[0064] As another optional implementation of the disclosed embodiment, when the image to be processed and the special effect image corresponding to the image to be processed are obtained, the image to be processed and the special effect image can be processed according to the special effect video generation algorithm to generate at least one generated image between the image to be processed and the special effect image. Further, a target special effect video with the image to be processed as the first frame and the special effect image as the last video frame can be generated according to the image to be processed, the special effect image and the at least one generated image.

[0065] It should be noted that in order to make the target special effects video generated finally fit more closely to the image to be processed and the special effects image, and to make the special effects of the target special effects video better, on the basis of generating the target special effects video based on the image to be processed and the special effects image, pre-configured special effects processing parameters can also be added. Furthermore, the target special effects video can be generated based on the image to be processed, the special effects image and the pre-configured special effects processing parameters.

[0066] It should also be noted that the target effect information corresponding to the target special effects applied to convert the image to be processed into a special effects image and generate a target special effects video including the image to be processed and the special effects image can be information pre-configured by the developer during the application software development phase; or, the target special effects information can also be information customized by the user based on the user's special effects editing requirements.

[0067] S130: Display the target special effects video.

[0068] In the embodiment of the present disclosure, when the target special effects video is obtained, the target special effects video can be displayed. Among them, the implementation methods of displaying the target special effects video may include multiple, optionally, directly displaying the target special effects video when it is generated; or, displaying the target special effects video in response to a video display trigger operation. The video display trigger operation may include at least one of the following operation methods: triggering a preset video display control; the received audio information includes a trigger word associated with the video display operation; receiving a preset video display instruction, etc.

[0069] As an optional implementation of the first embodiment of the present disclosure, when a target special effect video is generated, a video display operation can be triggered. Further, the target special effect video can be displayed in the display interface.

[0070] As another optional implementation of the disclosed embodiment, when a target special effects video is obtained and a video display trigger operation for the target special effects video is received, the target special effects video is displayed based on a display interface in response to the video display trigger operation.

[0071] The technical solution of the disclosed embodiment determines a special effect image corresponding to the image to be processed in response to the input operation of the target user for uploading the image to be processed; wherein, there is a difference in the style types of the special effect image and the image to be processed, and the effect of the special effect image with different style types is determined through a simple interactive operation, which provides a video generation basis for the subsequent generation of special effect videos; further, based on the image to be processed and the special effect image, a target special effect video is generated, wherein the image to be processed and the special effect image are video frames in the target special effect video; then, the target special effect video is displayed, which solves the problems of low content richness and poor picture display effect of the special effect video in the related art, and realizes the processing of the received image into special effect images with different style types, and generates the effect of the special effect video according to the two images with different style types, thereby enhancing the richness of the special effect video content, improving the special effect display effect of the special effect video, improving the flexibility and intelligence of the special effect video generation method, and improving the special effect application experience.

[0072] Figure 2A flowchart of another video generation method provided by an embodiment of the present disclosure. The technical solution of this embodiment further refines the method for determining the special effect image based on the above embodiment. Optionally, determining the special effect image corresponding to the image to be processed includes: performing stylized processing on the image to be processed based on a pre-configured target style model to obtain the special effect image. The specific implementation method can be found in the description of this embodiment. Among them, the technical features that are the same or similar to the above embodiments are not repeated here. Figure 2 As shown, the method of this embodiment may specifically include:

[0073] S210 . In response to an input operation of a target user for uploading an image to be processed, stylizing the image to be processed based on a pre-configured target style model to obtain a special effect image; wherein the style types of the special effect image and the image to be processed are different.

[0074] Among them, the target style model can be a neural network model that takes an image as an input object to perform stylized processing on the image and outputs an image with a specific stylized special effect. The target style type can be a stylized processing model corresponding to a style special effect determined based on the style editing operation of the target user; or, it can also be a stylized processing model adapted to the scene information of the image to be processed. It should be noted that the target style model can be a neural network model that can process the image to be processed into a special effect image of a specific style type, and the specific style type can be a style type required by the target user or a style type that is adapted to the scene information of the image to be processed and is different from the style type of the image to be processed; or, the target style model can be a neural network model that can process the image to be processed into special effect images of multiple style types, and these multiple style types include the style type required by the target user or a style type that is adapted to the scene information of the image to be processed and is different from the style type of the image to be processed.

[0075] In the embodiment of the present disclosure, the target style model can be obtained by training a deep learning model based on a sample image and an expected stylized special effect image corresponding to the sample image. Before applying the target style model provided by the embodiment of the present disclosure, the pre-constructed deep learning model can be trained in a supervised or unsupervised manner. Before training the model, multiple training samples can be constructed to train the model based on multiple training samples. Optionally, the training process of the target style model can be: obtaining multiple training samples; wherein the training samples include sample images and expected stylized special effect images; for each training sample, the sample image in the training sample is input into the pre-constructed deep learning model to stylize the sample image based on the deep learning model, and output the actual stylized special effect image; based on the actual stylized special effect image and the expected stylized special effect image in the training sample, determine the loss value; based on the loss value, correct the model parameters in the deep learning model, and use the convergence of the loss function in the deep learning model as the training goal, so as to use the trained deep learning model as the target style model.

[0076] As an optional implementation of the first embodiment of the present disclosure, when the target style model is obtained through training, the target style model can be configured in the application software. Further, after obtaining the image to be processed uploaded by the target user, the image content analysis can be performed on the image to be processed to determine the scene information corresponding to the image to be processed. Further, the target style model corresponding to the scene information can be determined, and the image to be processed can be input into the determined target style model. Further, the image to be processed can be stylized based on the target style model to generate a special effect image, and the generated special effect image can be output.

[0077] S220: Generate a target special-effect video based on the image to be processed and the special-effect image, wherein the image to be processed and the special-effect image are video frames in the target special-effect video.

[0078] S230, displaying the target special effects video.

[0079] The technical solution of the disclosed embodiment is to obtain a special effect image by stylizing the image to be processed based on a pre-configured target style model in response to the input operation of the target user for uploading the image to be processed; wherein, there is a difference in the style types of the special effect image and the image to be processed, thereby achieving the effect of processing the received image into the special effect image based on the pre-configured style model, improving the generation efficiency of the special effect image, and making the special effect display effect of the special effect image finally generated better.

[0080] Figure 3A flowchart of another video generation method provided by an embodiment of the present disclosure. The technical solution of this embodiment further refines the method for generating a target special effects video based on the above-mentioned embodiment. Optionally, based on the image to be processed and the special effects image, a target special effects video is generated, including: inputting the image to be processed and the special effects image into a special effects video generation model, and generating a target special effects video with the image to be processed as the first special effects video frame and the special effects image as the last special effects video frame. For specific implementation methods, please refer to the description of this embodiment. Among them, technical features that are the same or similar to those of the above-mentioned embodiments are not repeated here. Figure 3 As shown, the method of this embodiment may specifically include:

[0081] S310: In response to an input operation of a target user for uploading an image to be processed, determining a special effect image corresponding to the image to be processed; wherein the style types of the special effect image and the image to be processed are different.

[0082] S320, inputting the image to be processed and the special effect image into a special effect video generation model to generate a target special effect video with the image to be processed as the first special effect video frame and the special effect image as the last special effect video frame.

[0083] Among them, the special effects video generation model can be a neural network model that takes at least two images as input objects to generate a special effects video based on the input objects. The special effects video generation model can be used to generate a special effects video with one image as the first special effects video frame and another image as the last special effects video frame based on at least two images. The special effects video generation model can be a deep learning model of any model structure. It should be noted that the special effects video generation model can be used to generate at least one generated image between the image to be processed and the special effects image, and generate a target special effects video based on the image to be processed, the special effects image and at least one generated image.

[0084] In the disclosed embodiment, the special effects video generation model can be obtained by training a deep learning model based on a sample image, a special effects image corresponding to the sample image, and an expected special effects video. Before applying the special effects video generation model provided by the disclosed embodiment, the pre-constructed deep learning model can be trained in a supervised or unsupervised manner. Before training the model, multiple training samples can be constructed to train the model based on the multiple training samples. Optionally, the training process of the special effects video generation model can be: obtaining multiple training samples; wherein the training samples include sample images, special effects images corresponding to the sample images, and expected special effects videos with the sample images as the first special effects video frame and the special effects image as the last special effects video frame; for each training sample, inputting the sample image and special effects image in the training sample into a pre-built deep learning model, so as to generate an actual special effects video based on the sample images and special effects images through the deep learning model, and outputting the generated actual special effects video; determining a loss value based on the actual special effects video and the expected special effects video in the training sample; correcting the model parameters in the deep learning model based on the loss value, and taking the convergence of the loss function in the deep learning model as the training goal, so as to use the trained deep learning model as the special effects video generation model.

[0085] When the special effects video generation model is obtained through training, the special effects video generation model can be deployed in the application software. Furthermore, when the image to be processed and the special effects image corresponding to the image to be processed are obtained, the image to be processed and the special effects image can be input into the special effects video generation model. Furthermore, the image to be processed and the special effects image can be processed based on the special effects video generation model to generate at least one generated image between the image to be processed and the special effects image. Furthermore, a target special effects video with the image to be processed as the first special effects video frame and the special effects image as the last special effects video frame can be generated based on the image to be processed, the special effects image and at least one generated image.

[0086] S330, display the target special effects video.

[0087] The technical solution of the disclosed embodiment, by inputting the image to be processed and the special effect image into the special effect video generation model, generates a target special effect video with the image to be processed as the first special effect video frame and the special effect image as the last special effect video frame, thereby achieving the effect of generating a special effect video based on two images of different style types through a pre-configured special effect video generation model, improving the generation efficiency of the special effect video, and making the special effect display effect of the special effect video finally generated better.

[0088] Figure 4This is a flow chart of another video generation method provided by the embodiment of the present disclosure. The technical solution of this embodiment can pre-edit the target effect information based on the above embodiment, and then, when receiving the image to be processed, the target special effect video can be generated based on the target effect information. The specific implementation method can refer to the description of this embodiment. Among them, the technical features that are the same or similar to the above embodiment are not repeated here. Figure 4 As shown, the method of this embodiment may specifically include:

[0089] S410, displaying a special effect editing page.

[0090] The special effect editing page may be a visual editing page that supports users to make special effects. The special effect editing page may include multiple effect parameter editing controls. The effect parameter editing controls may be used to edit corresponding special effect effect parameters. Optionally, the effect parameter editing controls may include at least one basic parameter editing control and a stylized image generation control. The basic parameter editing control may be used to edit special effect processing parameters associated with special effect processing operations. The stylized image generation control may be used to edit special effect parameters associated with a stylized image generation process.

[0091] In an embodiment of the present disclosure, a method for triggering display of a special effects editing page may include at least one of the following: triggering a preset special effects editing control; the received audio information includes a trigger word associated with displaying the special effects editing page; receiving a preset special effects editing page display instruction, etc.

[0092] As an optional implementation of the first embodiment of the present disclosure, when a control trigger operation for a preset special effect editing control is detected, the control trigger operation can be responded to and the special effect editing page can be displayed. In order to facilitate special effect production and special effect editing based on the special effect editing page, at least one basic parameter editing control and a stylized image generation control can be displayed in the displayed special effect editing page. Furthermore, special effect editing can be performed based on the at least one basic parameter editing control and stylized image generation control displayed in the special effect editing page.

[0093] S420. Determine target effect information based on the editing operation on the special effect editing page, so that when receiving the image to be processed, generate a target special effect video based on the target effect information.

[0094] Among them, the target effect information at least includes effect parameters for generating a target special effect video including a special effect image and the image to be processed after the image to be processed is converted into a special effect image. The target effect information can be understood as a target special effect for converting the image to be processed into a special effect image, and generating a target special effect video based on the image to be processed and the special effect image. The effect parameters included in the target effect information can be special effect parameters that can convert the image to be processed into a special effect image, and generate a target special effect video including the special effect image and the image to be processed. Optionally, the target effect information may include at least a target style model for converting the image to be processed into a special effect image and a special effect video generation model for generating a target special effect video including the image to be processed and the special effect image.

[0095] In the embodiment of the present disclosure, when the special effect editing page is displayed, an editing operation can be input for at least one effect parameter editing control displayed in the special effect editing page. Further, the target effect information can be determined according to the input editing operation. Furthermore, when the image to be processed is received, the image to be processed can be converted into a special effect image according to the target effect information, and a target special effect video including the image to be processed and the special effect image is generated.

[0096] Optionally, the target special effect information includes partial effect information, target style model and special effect video generation model edited for at least one basic parameter editing control; based on the editing operation on the special effect editing page, the target effect information is determined, including: in response to the trigger operation in the special effect editing page, at least one basic parameter editing control is displayed in the special effect editing page to determine partial effect information in the target effect information based on the edited content in at least one basic parameter editing control; in response to the trigger operation of the stylized image generation control in the special effect editing page, a nested panel is displayed in the special effect editing page to determine the target style model and special effect video generation model in the target effect information based on the editing operation of at least one optional information in the nested panel. The advantage of such a setting is that the effect of determining the target effect information for generating a special effect video according to the interactive operation between the user and the special effect editing page is realized, the effect of custom editing and visual presentation of the target effect information is realized, the special effect editing operation is simplified, and the universality of the special effect editing operation is improved.

[0097] Among them, the basic parameter editing control can be understood as an interactive operation entry for editing the effect parameters in the target effect information. The basic parameter editing control can be used to edit the special effect processing parameters associated with the special effect processing operation. Optionally, the basic parameter editing control includes but is not limited to the video prompt information editing control and the camera movement mode editing control. The editing content in the basic parameter editing control can be the information displayed in the basic parameter editing control. Part of the effect information is a special effect processing parameter composed of the editing content in at least one basic parameter editing control. The stylized image generation control can be a control that triggers the entry into the stylized image generation process. The nested panel can be understood as a parameter editing panel nested in the stylized image generation control, and the parameter editing panel can be used to provide an interactive operation interface for editing at least one parameter associated with generating a stylized image. In general, when a trigger operation of any control is received, in response to the trigger operation, the parameter editing panel associated with the control is displayed, and the displayed parameter editing panel can be used as a nested panel corresponding to the control. The nested panel can be displayed in the special effect editing page in any form, optionally, a menu list display and / or a pop-up window display, etc. Exemplarily, when a trigger operation for a stylized image generation control is received, a nested panel can be displayed in the form of a menu list at any position of the special effect editing page (for example, on the left side of the interface or in the middle of the interface). Alternatively, when a trigger operation for a stylized image generation control is received, a nested panel is displayed in the special effect editing page in the form of a pop-up window. The optional information can be understood as a parameter editing control that is editable and displayed on the nested panel. The target style model can be used to process the image to be processed into a special effect image. The special effect video generation model can be used to generate a target special effect video.

[0098] It should be noted that in order to visually see the special effects of the target effect information during the target effect information editing process, a first image used as a special effects editing example image can also be obtained and displayed in the special effects editing page.

[0099] As an optional implementation of an embodiment of the present disclosure, in the case of receiving an editing operation for a special effect editing page, at least one basic parameter editing control can be displayed in the special effect editing page. Further, a parameter editing operation can be input for at least one basic parameter editing control, the editing content can be determined according to the parameter editing operation, and the editing content can be displayed in the corresponding basic parameter editing control. Further, partial effect information in the target effect information can be determined according to the editing content in at least one basic parameter editing control. Further, in the case of receiving a trigger operation for a stylized image generation control, a nested panel can be displayed in the special effect editing page, and at least one optional information is displayed in the nested panel. Further, an editing operation can be input for at least one optional information in the nested panel, and in the case of receiving an editing operation for the optional information, a target style model for processing the image to be processed into a special effect image and a special effect video generation model for generating a target special effect video can be determined according to the editing operation. Furthermore, partial effect information, target style model and special effect video generation model can be used as target effect information.

[0100] For example, Figure 5 This is a schematic diagram of the special effects editing page provided by the embodiment of the present disclosure. Figure 5 As shown, the special effect editing page 50 includes a video generation control 51, a display area 52 for the first image, three basic parameter editing controls 53, and a stylized image generation control 54. The basic parameter editing controls may include a video prompt word editing control, a shielding word editing control, and a camera movement editing control. Further, in the case of receiving a trigger operation for the stylized image generation control 54, a nested panel 55 may be displayed in the middle area of ​​the special effect editing page 50. Two optional information may be displayed in the nested panel 55, namely, an existing special effect preview control and an effect image generation control. The special effect editing page 50 also includes a target position 56 for displaying a style image.

[0101] In the disclosed embodiment, the optional information includes an existing special effect preview control and an effect image generation control. Furthermore, the target style model and the special effect video generation model may be determined in at least two ways: determining the target style model and the special effect video generation model based on the editing operation of the existing special effect preview control; or determining the target style model and the special effect video generation model based on the editing operation of the effect image generation control.

[0102] Optionally, based on the editing operation of at least one optional information in the nested panel, the target style model and the special effect video generation model in the target effect information are determined, including: in response to the triggering operation of the existing special effect preview control in the nested panel, at least one effect image to be selected is displayed, so as to determine the style image based on the triggered target effect image and the first image and display it at the target position; in response to the triggering operation of the application control, the target style model and the special effect video generation model of the triggering selection are determined. The advantage of such a setting is that the effect of determining the target effect information based on the editing of the existing special effect is realized, thereby improving the editing efficiency of the target effect information.

[0103] Among them, the existing special effect preview control can be an interactive operation entrance for special effect editing based on the existing special effect. The effect image to be selected can be a pre-generated special effect image to be selected. The effect image to be selected is an image obtained after processing based on the corresponding style model. In other words, an effect image to be selected can correspond to a style model. The effect image to be selected can be a template image for displaying the processing effect of the corresponding style model. The target effect image can be a to-be-selected effect image that is triggered and selected from at least one to-be-selected effect image. The first image can be an image to be processed with special effects. Optionally, the first image can be a default template image, or an image captured based on a camera device set on a terminal device, or an image obtained from a target storage space (such as an image library of an application software, or a local terminal album, etc.) in response to a user's trigger operation, or an image received from an external device and uploaded, or an image extracted from a video, etc. The style image can be an image obtained after the first image is processed with special effects based on the style model corresponding to the target effect image. The target position can be a display position in the special effect editing page for displaying effect preview information. The target position can be any position in the special effect editing page, optionally, the left side of the page, the middle of the page, or the right side of the page.

[0104] As an optional implementation of an embodiment of the present disclosure, in the case of receiving a trigger operation for an existing special effect preview control, in response to the trigger operation, at least one effect image to be selected is displayed based on the nested panel. Further, in the case of receiving a selection trigger operation for any effect image to be selected, the effect image to be selected that triggers the selection can be used as the target effect image. Further, the first image can be processed according to a style model corresponding to the target effect image to obtain a style image, and the style image can be displayed at a target position. Further, in the case of receiving a trigger operation for an application control, in response to the trigger operation, the style model corresponding to the target effect image is used as the target style model, and the special effect video generation model that triggers the selection is determined.

[0105] Optionally, based on the editing operation of at least one optional information in the nested panel, the target style model and the special effect video generation model in the target effect information are determined, including: in response to the triggering operation of the effect image generation control in the nested panel, at least one selectable style model is displayed to determine the style image based on the triggered target style model and the first image and display it at the target position; in response to the triggering operation of the application control, the target style model and the special effect video generation model of the trigger selection are determined. The advantage of such a setting is that the effect of determining the target effect information based on the existing style model editing is realized, the editing efficiency of the target effect information is improved, and the target effect information is determined by the above two methods, which enriches the determination method of the target effect information and improves the interest of the special effect editing process.

[0106] The effect image generation control may be an interactive operation entry for generating a special effect image. The selectable style model may be a pre-trained style model that can be selected by a user. The target style model may be a selectable style model that is triggered and selected from at least one selectable style model. The style image may be a special effect image obtained by performing special effect processing on the first image based on the target style model.

[0107] As another optional implementation of the disclosed embodiment, in the case of receiving a trigger operation for an effect image generation control, in response to the trigger operation, at least one selectable style model is displayed based on the nested panel. Further, in the case of receiving a selection trigger operation for any selectable style model, the selectable style model that triggers the selection can be used as the target style model. Furthermore, the first image can be processed with special effects according to the target style model to obtain a style image, and the style image can be displayed at the target position. Further, in the case of receiving a trigger operation for an application control, in response to the trigger operation, the target style model and the special effect video generation model that triggers the selection are determined.

[0108] It should be noted that the number of first images can be one or more. When the number of first images is one, the target effect information determined according to the editing operation of the special effect editing page can be used as the target effect information of the first image; when the first image includes multiple, the corresponding target effect information is configured for each first image, so that when the image to be processed is received, the target special effect video is determined based on the target effect information corresponding to the image to be processed. That is to say, when the first image includes multiple, the corresponding target effect information can be configured for each first image respectively. Further, the scene information corresponding to each first image is determined respectively, and the scene identifier corresponding to the scene information is associated with the target effect information and stored. Further, when the image to be processed is received, the scene information of the image to be processed can be obtained, and the corresponding target effect information can be retrieved according to the scene identifier corresponding to the scene information, so that the image to be processed is processed based on the target effect information to generate a target special effect video.

[0109] In the disclosed embodiment, before determining the style image, in order to make the final style image closer to the user's special effect editing requirements, the screen display parameters can also be edited. Then, the style image can be determined based on the edited screen display parameters and displayed at the target position.

[0110] Optionally, before determining the style image and displaying it at the target position, the method also includes: displaying at least one stylized parameter editing control corresponding to the triggered target effect image or the triggered target style model in the nested panel; and determining the screen display parameters in the target effect information based on the editing information in the at least one stylized parameter editing control.

[0111] Among them, the stylized parameter editing control may be an interactive operation entry for editing the corresponding stylized processing parameters. The at least one stylized parameter editing control corresponding to the target effect image may include but is not limited to a picture keyword editing control, a creative relevance editing control, a picture similarity editing control, a posture control editing control, etc. The at least one stylized parameter editing control corresponding to the target style model may include but is not limited to a picture keyword editing control, an intensity editing control, etc. The editing information may be information displayed in the corresponding stylized parameter editing control. The picture display parameter may be understood as the display parameter based on which the style image is determined.

[0112] As an optional implementation of an embodiment of the present disclosure, in the case of determining the target effect image that triggers the selection, at least one stylized parameter editing control corresponding to the target effect image can be displayed based on the nested panel, and the displayed stylized parameter editing control can include a picture keyword editing control, a creative relevance editing control, a picture similarity editing control, and a gesture control editing control. Further, in the case of receiving an editing operation for at least one stylized parameter editing control, the editing information corresponding to the stylized parameter editing control can be determined according to the editing operation, and the editing information can be displayed in the corresponding stylized parameter editing control. Further, when the editing completion operation is received, the picture display parameters corresponding to the target effect image in the target effect information can be determined according to the editing information in at least one stylized parameter editing control.

[0113] For example, Figure 6 This is a schematic diagram of an interface for determining a target style model and a special effect video generation model provided by an embodiment of the present disclosure. Figure 6 As shown, in the case of receiving the editing operation for the video prompt word editing control, the shielding word editing control and the camera movement mode editing control, partial effect information can be determined according to the editing operation, and the determined partial effect information can include the video prompt word, the shielding word and the horizontal camera movement mode. Further, in the case of receiving the trigger operation for the stylized image generation control, the nested panel 55 can be displayed. Further, in the case of receiving the trigger operation for the existing special effect preview control in the nested panel 55, three effect images to be selected can be displayed, namely, the effect image to be selected A, the effect image to be selected B and the effect image to be selected C. Further, in the case of receiving the selection trigger operation for the effect image to be selected B, the effect image to be selected B can be used as the target effect image, and the stylized parameter editing controls corresponding to the target effect image are displayed, namely, the picture keyword editing control, the shielding word editing control, the creative relevance editing control, the picture similarity editing control and the posture control editing control. Further, the picture display parameters can be determined according to the editing operation for the stylized parameter editing control. Then, the first image is processed according to the target effect image and the picture display parameters to obtain the style image, and the style image is displayed at the target position 56. Further, when a trigger operation for a "confirm and apply" control is received, target effect information may be determined.

[0114] As another optional implementation of the disclosed embodiment, when the target style model that triggers the selection is determined, at least one stylized parameter editing control corresponding to the target style model can be displayed based on the nested panel, and the displayed stylized parameter editing control can include a picture keyword editing control and an intensity editing control. Further, when an editing operation for at least one stylized parameter editing control is received, the editing information corresponding to the stylized parameter editing control can be determined according to the editing operation, and the editing information can be displayed in the corresponding stylized parameter editing control. Further, when the editing completion operation is received, the picture display parameters corresponding to the target style model in the target effect information can be determined according to the editing information in at least one stylized parameter editing control.

[0115] For example, Figure 7 This is a schematic diagram of an interface for determining a target style model and a special effect video generation model provided by an embodiment of the present disclosure. Figure 7 As shown, in the case of receiving the editing operation for the video prompt word editing control, the shielding word editing control and the camera movement mode editing control, partial effect information can be determined according to the editing operation, and the determined partial effect information can include the video prompt word, the shielding word and the horizontal camera movement mode. Further, in the case of receiving the trigger operation for the stylized image generation control, the nested panel 55 can be displayed. Further, in the case of receiving the trigger operation for the effect image generation control in the nested panel 55, three selectable style models can be displayed, namely, selectable style model A, selectable style model B and selectable style model C. Further, in the case of receiving the selection trigger operation for the selectable style model C, the selectable style model C can be used as the target style model, and the stylized parameter editing controls corresponding to the target style model are displayed, namely, the picture keyword editing control, the shielding word editing control and the intensity editing control. Further, the picture display parameters can be determined according to the editing operation for the stylized parameter editing control. Then, the first image is processed according to the target style model and the picture display parameters to obtain the style image, and the style image is displayed at the target position 56. Further, when a trigger operation for a "confirm and apply" control is received, target effect information may be determined.

[0116] S430: In response to the input operation of the target user for uploading the image to be processed, determining a special effect image corresponding to the image to be processed; wherein the style types of the special effect image and the image to be processed are different.

[0117] S440: Generate a target special-effect video based on the image to be processed and the special-effect image, wherein the image to be processed and the special-effect image are video frames in the target special-effect video.

[0118] S450, display the target special effects video.

[0119] For example, Figure 8 FIG. 1 is a flow chart of a method for determining target effect information provided by an embodiment of the present disclosure. Figure 8 As shown, the process of determining the target effect information can be: first, display the special effect editing page, and determine the special effect to be edited based on the trigger operation of the special effect editing page. Further, determine whether the special effect editing is allowed according to the business field corresponding to the special effect to be edited, and if it is determined that the special effect editing is allowed, display at least one basic parameter editing control and a stylized image generation control in the special effect editing page. Further, based on the edited content in at least one basic parameter editing control, determine the partial effect information in the target effect information, and display the nested panel when receiving the trigger operation for the stylized image generation control. Further, based on the edit operation of at least one optional information in the nested panel, determine the target style model and the special effect video generation model in the target effect information, and determine the screen display parameters in the target effect information according to the edit information in at least one stylized parameter editing control corresponding to the target style model. Further, when receiving the trigger operation for the application control, the target effect information can be determined according to the partial effect information, the target style model, the special effect video generation model and the screen display parameters.

[0120] Fig. 9 A flow chart of a video generation method provided by an embodiment of the present disclosure. Fig. 9 As shown, in the case where the desktop receives an application operation for the target effect information, a workflow can be constructed according to the target effect information, and the workflow can be sent to the special opening server. Further, in the case where the special opening server receives the workflow, a workflow request can be sent to the cloud editing server based on the special opening server. Further, in the case where the cloud editing server receives the workflow request, the information identifier of the target effect information can be obtained according to the workflow request, and the corresponding target effect information can be requested from Loki to parse the workflow. Further, the cloud editing server calls the scene recognition algorithm to detect and identify the scene information of the image to be processed, and after identifying the scene information, the cloud editing server determines the corresponding target style model according to the scene information, and sends the image to be processed and the target style model to the artificial intelligence platform to obtain the generated special effect image. Finally, the cloud editing server sends the generated special effect image and the special effect video generation model in the target effect information to the artificial intelligence platform to obtain the final generated target special effect video. Finally, the cloud editing server returns the generated target special effect video to the desktop through the special opening server, and the desktop is rendered on the screen.

[0121] The technical solution of the disclosed embodiment displays a special effect editing page; further, based on the editing operation on the special effect editing page, the target effect information is determined, so that when the image to be processed is received, the target special effect video is generated based on the target effect information. This achieves the effect of determining the target effect information for generating a special effect video based on the interaction between the user and the special effect editing page, achieves the effect of custom editing and visual presentation of the target effect information, simplifies the special effect editing operation, and improves the universality of the special effect editing operation.

[0122] Fig.10 A schematic diagram of the structure of a video generating device provided by an embodiment of the present disclosure is shown in FIG. Fig.10 As shown, the device includes: a special effect image generation module 510, a special effect video generation module 520 and a special effect video display module 530.

[0123] Among them, the special effect image generation module 510 is used to determine the special effect image corresponding to the image to be processed in response to the input operation of the target user for uploading the image to be processed; wherein there is a difference in the style type of the special effect image and the image to be processed; the special effect video generation module 520 is used to generate a target special effect video based on the image to be processed and the special effect image, wherein the image to be processed and the special effect image are video frames in the target special effect video; the special effect video display module 530 is used to display the target special effect video.

[0124] The technical solution of the disclosed embodiment determines a special effect image corresponding to the image to be processed in response to the input operation of the target user for uploading the image to be processed; wherein, there is a difference in the style types of the special effect image and the image to be processed, and the effect of the special effect image with different style types is determined through a simple interactive operation, which provides a video generation basis for the subsequent generation of special effect videos; further, based on the image to be processed and the special effect image, a target special effect video is generated, wherein the image to be processed and the special effect image are video frames in the target special effect video; then, the target special effect video is displayed, which solves the problems of low content richness and poor picture display effect of the special effect video in the related art, and realizes the processing of the received image into special effect images with different style types, and generates the effect of the special effect video according to the two images with different style types, thereby enhancing the richness of the special effect video content, improving the special effect display effect of the special effect video, improving the flexibility and intelligence of the special effect video generation method, and improving the special effect application experience.

[0125] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, the special effect image generation module 510 is specifically used to perform stylization processing on the image to be processed based on a pre-configured target style model to obtain the special effect image.

[0126] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, the special effect image generation module 510 includes: a feature extraction unit and a special effect image generation unit.

[0127] A feature extraction unit, used to extract image features of the image to be processed, so as to determine scene information of the image to be processed based on the image features;

[0128] The special effect image generation unit is used to retrieve a target style model that is compatible with the scene information to process the image to be processed to obtain the special effect image.

[0129] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, a special effects video generation module 520 is specifically used to input the image to be processed and the special effects image into a special effects video generation model to generate a target special effects video with the image to be processed as the first special effects video frame and the special effects image as the last special effects video frame.

[0130] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, the device further includes: a page display module and an effect information determination module.

[0131] Page display module, used to display special effects editing page;

[0132] An effect information determination module is used to determine target effect information based on the editing operation on the special effect editing page, so as to generate the target special effect video based on the target effect information when receiving the image to be processed; wherein the target effect information at least includes effect parameters for generating a target special effect video including a special effect image and the image to be processed after converting the image to be processed into a special effect image.

[0133] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, the effect information determination module includes: a partial effect information determination unit and a target effect information determination unit.

[0134] a partial effect information determining unit, configured to display at least one basic parameter editing control in the special effect editing page in response to a trigger operation in the special effect editing page, so as to determine partial effect information in the target effect information based on edited content in the at least one basic parameter editing control;

[0135] A target effect information determination unit is used to respond to a triggering operation on a stylized image generation control in the special effect editing page, and to display a nested panel in the special effect editing page, so as to determine a target style model and a special effect video generation model in the target effect information based on an editing operation of at least one optional information in the nested panel; wherein the target style model is used to process the image to be processed into a special effect image, and the special effect video generation model is used to generate a target special effect video.

[0136] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, the optional information includes an existing special effect preview control and an effect image generation control, and the target effect information determination unit includes: a style image first determination subunit, a style image second determination subunit and a video generation model determination subunit.

[0137] a style image first determination subunit, for displaying at least one effect image to be selected in response to a triggering operation on the existing special effect preview control in the nested panel, so as to determine the style image based on the triggered target effect image and the first image and display it at a target position, wherein the effect image to be selected is an image obtained after processing based on the corresponding style model; or,

[0138] a second style image determining subunit, configured to display at least one selectable style model in response to a triggering operation on the effect image generating control in the nested panel, so as to determine the style image and display it at the target position based on the triggered target style model and the first image;

[0139] The video generation model determination subunit is used to determine the target style model and special effect video generation model for triggering selection in response to a trigger operation on the application control.

[0140] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, the device further includes: an image display module and a screen display parameter determination module.

[0141] An image display module, configured to display at least one stylized parameter editing control corresponding to the triggered target effect image or the triggered target style model in the nested panel before the style image is determined and displayed at the target position;

[0142] The screen display parameter determination module is used to determine the screen display parameters in the target effect information based on the editing information in the at least one stylized parameter editing control.

[0143] On the basis of any optional technical solution in the embodiments of the present disclosure, optionally, when the first image includes multiple images, corresponding target effect information is configured for each of the first images, so that when the image to be processed is received, the target special effects video is determined based on the target effect information corresponding to the image to be processed.

[0144] The video generating device provided in the embodiments of the present disclosure can execute the video generating method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects for executing the video generating method.

[0145] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0146] Reference below Fig.11 , which shows a schematic diagram of the structure of an electronic device (such as a terminal device or a server) 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0147] like Fig.11 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0148] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0149] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0150] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0151] The electronic device provided in the embodiment of the present disclosure and the video generation method provided in the above embodiment belong to the same inventive concept. The technical details not fully described in the embodiment of the present disclosure can be referred to the above embodiment, and the present embodiment has the same beneficial effects as the above embodiment.

[0152] The embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the video generation method provided by the above embodiment is implemented.

[0153] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0154] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0155] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0156] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to the input operation of the target user for uploading the image to be processed, determines the special effect image corresponding to the image to be processed; wherein there is a difference in the style type of the special effect image and the image to be processed; based on the image to be processed and the special effect image, generates a target special effect video, wherein the image to be processed and the special effect image are video frames in the target special effect video; and displays the target special effect video.

[0157] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0158] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, a special effect image generation module may also be described as a "module for determining a special effect image corresponding to an image to be processed".

[0160] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0161] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0162] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0163] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0164] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video generation method, characterized in that: include: In response to an input operation of a target user for uploading an image to be processed, determining a special effect image corresponding to the image to be processed; wherein the style types of the special effect image and the image to be processed are different; Generate a target special-effect video based on the image to be processed and the special-effect image, wherein the image to be processed and the special-effect image are video frames in the target special-effect video; Display the target special effects video.

2. The method according to claim 1, characterized in that The determining of the special effect image corresponding to the image to be processed includes: The image to be processed is stylized based on a pre-configured target style model to obtain the special effect image.

3. The method according to claim 1 or 2, characterized in that: The determining of the special effect image corresponding to the image to be processed includes: Extracting image features of the image to be processed to determine scene information of the image to be processed based on the image features; A target style model adapted to the scene information is retrieved to process the image to be processed to obtain the special effect image.

4. The method according to claim 1, characterized in that: The step of generating a target special effect video based on the image to be processed and the special effect image comprises: The image to be processed and the special effect image are input into a special effect video generation model to generate a target special effect video with the image to be processed as the first special effect video frame and the special effect image as the last special effect video frame.

5. The method according to claim 1, characterized in that The method further comprises: Display the special effects editing page; Based on the editing operation on the special effect editing page, target effect information is determined, so that when the image to be processed is received, the target special effect video is generated based on the target effect information; The target effect information at least includes effect parameters of a target special effect video including a special effect image and the image to be processed, which is generated after the image to be processed is converted into a special effect image.

6. The method according to claim 5, characterized in that The determining target effect information based on the editing operation on the special effect editing page includes: In response to a trigger operation in a special effect editing page, displaying at least one basic parameter editing control in the special effect editing page, so as to determine part of the effect information in the target effect information based on the edited content in the at least one basic parameter editing control; In response to a triggering operation of a stylized image generation control in the special effect editing page, displaying a nested panel in the special effect editing page to determine a target style model and a special effect video generation model in the target effect information based on an editing operation of at least one optional information in the nested panel; Among them, the target style model is used to process the image to be processed into a special effect image, and the special effect video generation model is used to generate a target special effect video.

7. The method according to claim 6, characterized in that The optional information includes an existing special effect preview control and an effect image generation control, and the editing operation based on at least one optional information in the nested panel to determine the target style model and the special effect video generation model in the target effect information includes: In response to a triggering operation on the existing special effect preview control in the nested panel, at least one effect image to be selected is displayed, so as to determine a style image based on the triggered target effect image and the first image and display it at a target position, wherein the effect image to be selected is an image obtained after processing based on a corresponding style model; or, In response to a triggering operation on the effect image generation control in the nested panel, at least one selectable style model is displayed to determine a style image based on the triggered target style model and the first image and to display the style image at a target position; In response to a trigger operation on an application control, a target style model and a special effect video generation model for triggering selection are determined.

8. The method according to claim 7, characterized in that Before determining the style image and displaying it at the target location, the method further includes: Displaying at least one stylized parameter editing control corresponding to the triggered target effect image or the triggered target style model in the nested panel; Based on the editing information in the at least one stylized parameter editing control, the screen display parameters in the target effect information are determined.

9. The method according to claim 7, characterized in that: In the case where the first images include multiple ones, corresponding target effect information is configured for each of the first images, so that when an image to be processed is received, the target special effect video is determined based on the target effect information corresponding to the image to be processed.

10. A video generating device, characterized in that: include: A special effect image generation module, configured to determine, in response to an input operation of a target user for uploading an image to be processed, a special effect image corresponding to the image to be processed; wherein the style types of the special effect image and the image to be processed are different; A special effect video generation module, used to generate a target special effect video based on the image to be processed and the special effect image, wherein the image to be processed and the special effect image are video frames in the target special effect video; The special effects video display module is used to display the target special effects video.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the video generating method as described in any one of claims 1-9.

12. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the video generation method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the video generation method according to any one of claims 1 to 9.

Images