Special effect processing method, apparatus, device, medium, and product

Abstract

Embodiments of the present disclosure provide a special effect processing method and device, electronic equipment, storage medium and program product. The method comprises: in response to a special effect processing request, obtaining a to-be-processed image, wherein the to-be-processed image comprises a target foreground object; performing special effect processing on the to-be-processed image to obtain a target special effect video, and displaying the target special effect video; wherein the target special effect video comprises a target foreground image of the to-be-processed image; the target foreground image comprises the target foreground object, and at least part of an object region of the target foreground object is presented in a preset style. The technical scheme of the embodiments of the present disclosure can automatically identify and extract the target foreground object in the to-be-processed image in the special effect processing process, and perform stylization processing on the target foreground object, thereby realizing differentiated processing of the target foreground object in the to-be-processed image, enriching the special effect processing mode, increasing the content level of the image after special effect processing, and improving the special effect interaction experience.

Description

Technical Field

[0001] This disclosure relates to the field of computer processing technology, and more particularly to a special effects processing method, apparatus, device, medium, and product. Background Technology

[0002] In image processing and video production scenarios, special effects are highly favored by users. By applying selected special effects to process images or videos, their presentation can be enriched.

[0003] In related technologies, adding special effects to images typically involves adding preset effect elements (such as stickers) to create the desired effect. However, this method of processing effects results in a rather monotonous effect, negatively impacting the user experience. Summary of the Invention

[0004] This disclosure provides a special effects processing method, apparatus, device, medium, and product to automatically identify and extract target foreground objects in an image to be processed during the special effects processing process, and to perform stylization processing on the target foreground objects to obtain a stylized target foreground image, thereby generating a target special effects video containing the target foreground image.

[0005] In a first aspect, embodiments of this disclosure provide a special effects processing method, the method comprising:

[0006] In response to a special effects processing request, an image to be processed is obtained, wherein the image to be processed includes a target foreground object;

[0007] The image to be processed is subjected to special effects processing to obtain a target special effects video, and the target special effects video is displayed; wherein, the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0008] Secondly, this disclosure also provides a special effects processing apparatus, which includes:

[0009] The special effects processing request module is used to respond to a special effects processing request and obtain an image to be processed, wherein the image to be processed includes a target foreground object;

[0010] A special effects video display module is used to perform special effects processing on the image to be processed to obtain a target special effects video and display the target special effects video; wherein, the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0011] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0012] One or more processors;

[0013] Storage device for storing one or more programs.

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects processing method as described in any of the embodiments of this disclosure.

[0015] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform special effects processing methods as described in any of the embodiments of this disclosure.

[0016] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the special effects processing method as described in any of the embodiments of this disclosure.

[0017] The technical solution of this disclosure, in response to a special effects processing request, acquires an image to be processed. The acquisition of the image to be processed can be triggered through a simple interactive operation. Furthermore, the image to be processed includes a target foreground object, providing a data foundation for subsequent special effects processing of the foreground object in the image. Further, special effects processing is applied to the image to be processed to obtain a target special effects video, which is then displayed. The target special effects video includes a target foreground image of the image to be processed. The target foreground image includes a target foreground object, and at least a portion of the object area of ​​the target foreground object is presented in a preset style. This solves the problem of monotonous and uninteresting presentation of image special effects in related technologies. During the special effects processing, the target foreground object in the image to be processed can be automatically identified and extracted, and stylized to obtain a stylized target foreground image. Subsequently, a target special effects video containing the target foreground image is generated. This achieves differentiated processing of the target foreground object in the image to be processed, enriches the special effects processing methods, increases the content hierarchy of the processed image, and enhances the interactive experience of the special effects. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0019] Figure 1A schematic flowchart illustrating a special effects processing method provided in an embodiment of this disclosure;

[0020] Figure 2 A flowchart illustrating another special effects processing method provided in an embodiment of this disclosure;

[0021] Figure 3 A flowchart illustrating another special effects processing method provided in an embodiment of this disclosure;

[0022] Figure 4 A flowchart illustrating another special effects processing method provided in an embodiment of this disclosure;

[0023] Figure 5 This is a schematic diagram illustrating an interface of the special effects editing process provided in an embodiment of the present disclosure.

[0024] Figure 6 A schematic flowchart illustrating a special effects processing method provided in an embodiment of this disclosure;

[0025] Figure 7 This is a schematic diagram of the structure of a special effects processing device provided in an embodiment of the present disclosure;

[0026] Figure 8 This is a schematic diagram of the structure of an electronic device for implementing an embodiment of the present disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0034] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0037] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0038] Figure 1This is a flowchart illustrating a special effects processing method provided in an embodiment of this disclosure. This embodiment is applicable to scenarios involving special effects processing of acquired images to be processed. The method can be executed by a special effects processing device, which can be implemented in software and / or hardware, optionally through an electronic device, such as a mobile terminal, PC, or server. Figure 1 As shown, the method in this embodiment may specifically include:

[0039] S110. In response to the special effects processing request, obtain the image to be processed, wherein the image to be processed includes the target foreground object.

[0040] In this context, a special effects processing request can be understood as an instruction for performing special effects processing on an image. Special effects processing requests can include various generation methods. Optionally, a special effects processing request can be generated when a trigger operation targeting a preset special effects processing control is detected; or, a special effects processing request can be generated when a trigger word associated with special effects processing is detected in received audio information; or, a special effects processing request can be generated when a received instruction includes a special effects processing instruction. The image to be processed can be an image to be processed with special effects. Optionally, the image to be processed can be a default template image, an image captured by a camera device set on a terminal device, an image obtained from a target storage space (such as an application software image library, or a local terminal photo album, etc.) in response to a user's trigger operation, or an image uploaded by an external device, etc. In this embodiment, the image to be processed includes a target foreground object. A target foreground object can be understood as an object determined after detecting the image to be processed using a preset foreground object detection algorithm. That is, objects included in the foreground image of the image to be processed can be used as target foreground objects. It should be noted that all objects included in the foreground image of the image to be processed can be used as the target foreground object; or, at least one object can be selected from all objects included in the foreground image of the image to be processed according to a preset object selection logic, and the selected at least one object can be used as the target foreground object. The target foreground object can be any type of object, and can optionally include people, animals, buildings, or preset items, etc.

[0041] It should be noted that when the image to be processed includes a target foreground object, it indicates that the image to be processed may include a foreground image. In the embodiments of this disclosure, the image to be processed may only include a foreground image; or, the image to be processed may also include a foreground image and a background image. The background image may be any type of background image. Optionally, it may be the original background image acquired at the same time as the foreground image; or it may be a preset background image, such as a pure white background, a pure black background, or a background image extracted from other images.

[0042] As an optional implementation of this embodiment, a special effects processing request can be generated upon detecting a control trigger operation input to a preset special effects processing control. Further, in response to the special effects processing request, an image display interface is displayed, which includes at least one candidate image. Further, upon detecting an image selection operation on a candidate image, a selected candidate image can be determined based on the image selection operation, and the selected candidate image can be used as the image to be processed. Furthermore, the main object included in the foreground image of the image to be processed can be used as the target foreground object.

[0043] S120. Perform special effects processing on the image to be processed to obtain a target special effects video and display the target special effects video, wherein the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes a target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0044] The target effects video can be a special effects video used to present the special effects processing effect of the image to be processed. The target effects video includes a target foreground image of the image to be processed; the target foreground image includes a target foreground object, and at least a portion of the object area of ​​the target foreground object is presented in a preset style. The target foreground image can be the foreground image in the special effects image obtained after processing the image to be processed. At least a portion of the object area of ​​the target foreground object included in the target foreground image is presented in a preset style. The at least a portion of the object area can be all and / or part of the object area in the target foreground object. For example, if the target foreground object is a person, the at least a portion of the object area can be the display area of ​​accessories and / or clothing worn by the person; if the target foreground image is an animal, the at least a portion of the object area can be the display area of ​​the animal's fur; if the target foreground image is a preset item, the at least a portion of the object area can be the entire object area of ​​the preset item. The preset style can be a stylized effect used when stylizing the target foreground object. The preset style can be any style, optionally including a "fluffy" style, cartoon style, retro style, or cyberpunk style, etc. The "fluffy" style can be understood as presenting the target foreground object in an image with a texture similar to that of a plush toy.

[0045] In this embodiment of the disclosure, upon obtaining an image to be processed, special effects processing can be applied to the image to obtain a target foreground image. Further, a target special effects video can be determined based on the target foreground image and displayed. The determination of the target foreground image can include various methods, optionally including processing the image to be processed based on a stylization processing model to obtain the target foreground image; or stylizing the target foreground image in the image to be processed according to a stylization processing algorithm corresponding to a preset style, and generating the target foreground image based on the target foreground image presented in the preset style, etc.

[0046] It should be noted that, in the embodiments of this disclosure, the target special effects video can be a special effects video that presents a target foreground image as the video foreground information and any background information as the video background information; or, the target special effects video can also be a video that presents a target foreground image as the video foreground information and a video background image as the video background information. The video background image may include at least one of the following: the image to be processed, the target background image of the image to be processed, and a preset background image.

[0047] Optionally, when the target special effects video is presented with a target foreground image as the video foreground information and arbitrary background information as the video background information, after obtaining the image to be processed, special effects processing can be applied to the image to be processed to obtain the target foreground image, wherein at least a portion of the object area of ​​the target foreground object in the target foreground image is presented in a preset style. Further, the target special effects video can be determined based on the target foreground image and the preset video background information.

[0048] Optionally, when the target special effects video is presented with the target foreground image as the video foreground information and the video background image as the video background information, and given the image to be processed, special effects processing can be applied to the image to be processed to obtain the target foreground image and the video background image. Furthermore, the target special effects video can be generated based on the target foreground image and the video background image.

[0049] In the embodiments of this disclosure, the target special effects video can be a special effects video generated based on a single image to be processed, or it can be a special effects video generated based on multiple images to be processed.

[0050] Optionally, special effects processing is applied to the image to be processed to obtain a target special effects video, including: if there is an image to be processed, determining the image special effects video corresponding to the image to be processed, and using the determined image special effects video as the target special effects video.

[0051] Among them, image effects videos can be understood as special effects animations determined based on the images to be processed.

[0052] As an optional implementation of this disclosure, if the acquired image to be processed is a single image, special effects processing can be applied to the image to be processed to obtain a target foreground image. Furthermore, an image effects video can be determined based on the target foreground image, and this image effects video can be used as the target special effects video.

[0053] Optionally, special effects processing is applied to the images to be processed to obtain a target special effects video, including: when there are multiple images to be processed, determining the image special effects video corresponding to each image to be processed, and determining the target special effects video based on the image special effects videos corresponding to the multiple images to be processed.

[0054] As another optional implementation of this disclosure, when multiple images to be processed have been acquired, special effects processing can be performed on each of the multiple images to be processed to obtain multiple target foreground images. Further, the multiple target foreground images can be processed separately to obtain image effect videos corresponding to the multiple images to be processed. Further, multiple image effect videos can be spliced ​​together according to a preset video splicing order, and the spliced ​​effect video can be used as the target effect video. The video splicing order can include the image acquisition order of the images to be processed, the determination order of the image effect videos, a random splicing order, or other preset orders. The advantage of this setting is that it achieves the effect of splicing image effect videos corresponding to multiple images to be processed into a single target effect video, realizing the effect of performing special effects processing on multiple images to be processed to obtain a target effect video, enriching the methods for determining the target effect video, and enhancing the flexibility of the special effects processing methods.

[0055] In this embodiment of the disclosure, the target special effects video can be a video displaying static special effects or a video displaying dynamic special effects. When the target special effects video is displayed with static special effects, the images included in the target special effects video can be displayed with static effects. When the target special effects video is displayed with dynamic special effects, the images included in the target special effects video can be displayed with animated effects.

[0056] Optionally, if the target effects video includes a target foreground image, the target foreground image is displayed with a first animation effect; and if the target effects video includes a target background image of the image to be processed, the target foreground image is displayed with a first animation effect and the target background image is displayed with a second animation effect; and if the target effects video includes both the image to be processed and the target background image of the image to be processed, the target foreground image is displayed with a first animation effect, the target background image is displayed with a second animation effect, and the image to be processed is displayed with a third animation effect. The advantage of this setup is that it achieves the effect of displaying all images included in the target effects video with their corresponding animation effects, realizes layered effects processing of the image to be processed to obtain the effect of the target effects video, and enriches the effects display effect of the target effects video.

[0057] The first animation effect can be any animation effect, optionally including camera movement animation effects (e.g., the target foreground object changing from small to large or from large to small), or the target foreground object swaying in the target special effects video. The target background image can be the background image in the special effects image obtained after processing the image to be processed. The second animation effect can be any animation effect, optionally including camera movement animation effects (e.g., the target background image changing from small to large or the target foreground object changing from large to small), or the target background image swaying in the target special effects video. The third animation effect can be any animation effect, optionally including camera movement animation effects, or the image to be processed swaying in the target special effects video.

[0058] It should be noted that when the target effects video includes the target background image of the image to be processed, the first animation effect and the second animation effect can be the same or different animation effects. When the target effects video includes the image to be processed and the target background image of the image to be processed, the first animation effect, the second animation effect, and the third animation effect can be the same or different animation effects. For example, the first animation effect, the second animation effect, and the third animation effect can be completely different, or two of the first animation effect, the second animation effect, and the third animation effect can be different.

[0059] The technical solution of this disclosure, in response to a special effects processing request, acquires an image to be processed. The acquisition of the image to be processed can be triggered through a simple interactive operation. Furthermore, the image to be processed includes a target foreground object, providing a data foundation for subsequent special effects processing of the foreground object in the image. Further, special effects processing is applied to the image to be processed to obtain a target special effects video, which is then displayed. The target special effects video includes a target foreground image of the image to be processed. The target foreground image includes a target foreground object, and at least a portion of the object area of ​​the target foreground object is presented in a preset style. This solves the problem of monotonous and uninteresting presentation of image special effects in related technologies. During the special effects processing, the target foreground object in the image to be processed can be automatically identified and extracted, and stylized to obtain a stylized target foreground image. Subsequently, a target special effects video containing the target foreground image is generated. This achieves differentiated processing of the target foreground object in the image, enriches the special effects processing methods, increases the content hierarchy of the processed image, and enhances the interactive experience of the special effects.

[0060] Figure 2 This is a flowchart illustrating another special effects processing method provided in this embodiment. Based on the above embodiments, the technical solution of this embodiment presents the target foreground image as foreground information in the target special effects video, further refining the process of determining the target special effects video. Optionally, performing special effects processing on the image to be processed to obtain the target special effects video includes: inputting the image to be processed into a stylization processing model to obtain the target foreground image, and determining the target special effects video based on the target foreground image. Detailed implementation methods can be found in the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 2 As shown, the method in this embodiment may specifically include:

[0061] S210. In response to the special effects processing request, obtain the image to be processed, wherein the image to be processed includes the target foreground object.

[0062] S220. Input the image to be processed into the stylization processing model to obtain the target foreground image, determine the target special effects video based on the target foreground image, and display the target special effects video; wherein, the target special effects video includes the target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0063] In this context, video foreground information typically refers to display information located in a prominent position within the video frame and exhibiting animated effects. In simpler terms, foreground objects are usually unoccluded objects in the image to be processed. Video foreground information is generally displayed in the upper layer of the video frame. A stylization processing model can be a neural network model that takes an image as input, performs stylization processing on the image, and outputs an image with specific stylized effects. Stylization processing models can be used to extract foreground objects from an image and convert them to a preset style to generate the desired foreground image. Stylization processing models can be deep learning models with arbitrary model structures.

[0064] In this embodiment of the disclosure, the stylization processing model is obtained by training a first deep learning model based on a first sample image and a desired foreground image. Before applying the stylization processing model provided in this embodiment, the pre-built first deep learning model can be trained in a supervised or unsupervised manner. Before training the model, multiple training samples can be constructed to train the model based on these multiple training samples. Optionally, the training process of the stylization processing model can be as follows: acquiring multiple training samples; wherein the training samples include a first sample image and a desired foreground image; for each training sample, inputting the first sample image from the training sample into the pre-built first deep learning model to extract the sample foreground object in the first sample image based on the first deep learning model, converting the sample foreground object into a preset style to generate an actual foreground image, and outputting the actual foreground image; determining a loss value based on the actual foreground image and the desired foreground image in the training samples; correcting the model parameters in the first deep learning model based on the loss value, and using the convergence of the loss function in the first deep learning model as the training objective, so as to use the trained first deep learning model as the stylization processing model. The first sample image can be an image including sample foreground objects. The desired foreground image includes sample foreground objects from the first sample image, and the sample foreground objects are presented in a preset style.

[0065] In this embodiment of the disclosure, after obtaining the image to be processed, it can be input into a stylization processing model. Further, a target foreground object can be extracted from the image to be processed based on the stylization processing model, the target foreground object can be converted into a preset style, and a target foreground image can be generated. Then, the target foreground image is output. Further, a target special effects video can be determined based on the target foreground image, and the target special effects video can be displayed.

[0066] It should be noted that when the target foreground image is presented as video foreground information in the target special effects video, the video background information of the target special effects video can be any form of background information. This disclosure does not specifically limit the video background information of the target special effects video.

[0067] It should also be noted that there are several ways to determine the target special effects video based on the target foreground image. Optionally, the target foreground image can be displayed according to a preset display duration and / or display format, and the resulting video can be used as the target special effects video; or, at least one foreground generation image can be generated based on the target foreground image, and the target foreground image and at least one foreground generation image can be displayed according to a preset display duration and / or display format, and the resulting video can be used as the target special effects video; or, the target foreground image can be processed based on the animation video corresponding to the target foreground image to obtain the target special effects video, etc.

[0068] Optionally, determining the target special effects video based on the target foreground image includes: determining first animation data corresponding to the target foreground image; determining video foreground animation based on the target foreground image and the first animation data; and determining the target special effects video based on the video foreground animation. The advantage of this setup is that it allows the target foreground image to be presented with animation effects, further enriching the video foreground information of the target special effects video and improving the display effect of the video foreground information.

[0069] The first animation data can be understood as data that causes the applied image and / or video to be presented with a preset animation effect. In this embodiment of the disclosure, the first animation data may be animation data corresponding to the target foreground image, that is, the first animation data is predetermined animation data acting on the target foreground image. Video foreground animation can be understood as animated elements and / or dynamic effects in the foreground portion of the video frame. For example, the first animation data may be a combination of appearance animation and camera movement animation data. Furthermore, video foreground animation may be a target foreground object of a preset style appearing in the frame in an appearance manner corresponding to the appearance animation, and presenting a camera movement animation effect in the frame.

[0070] In the embodiments of this disclosure, the method for determining the first animation data corresponding to the target foreground image may include various methods. Optionally, it may be determined from at least one preset animation data based on the image type identifier corresponding to the target foreground image; or, the preset animation data corresponding to the foreground image may be directly used as the first animation data, etc.

[0071] As an optional implementation of this disclosure, multiple image types can be determined based on the object type of the foreground object, and animation data corresponding to each image type can be determined respectively. Further, after obtaining the target foreground image, an image type identifier corresponding to the target foreground image can be determined. Further, based on the image type identifier, animation data corresponding to the image type identifier can be determined from at least one preset animation data set, and this animation data can be used as the first animation data corresponding to the target foreground image. Further, the target foreground image can be processed according to the first animation data to obtain a target foreground animation. Further, a target special effects video can be generated based on the target foreground animation and preset video background information.

[0072] The technical solution of this disclosure embodiment obtains a target foreground image by inputting the image to be processed into a stylization processing model, and determines a target special effects video based on the target foreground image. This achieves the effect of stylizing the foreground object in the image to be processed to generate a target foreground image based on the stylization processing model. Then, the target special effects video is determined based on the generated target foreground image, so that the foreground object in the target special effects video is presented in a preset style, thereby enhancing the special effects display effect of the target special effects video.

[0073] Figure 3 This is a flowchart illustrating another special effects processing method provided in this embodiment. Based on the above embodiments, the technical solution of this embodiment presents a target foreground image as video foreground information in the target special effects video; the video background information of the target special effects video presents a video background image; the video background image includes at least one of the following: an image to be processed, a target background image of the image to be processed, and a preset background image. The process of determining the target special effects video is further refined. Optionally, special effects processing is applied to the image to be processed to obtain the target special effects video, including: inputting the image to be processed into a background extraction model to obtain a target background image, and determining the target special effects video based on the target foreground image and the target background image. Specific implementation details can be found in the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 3 As shown, the method in this embodiment may specifically include:

[0074] S310. In response to the special effects processing request, obtain the image to be processed, wherein the image to be processed includes the target foreground object.

[0075] S320. Determine the target foreground image of the image to be processed, and input the image to be processed into the background extraction model to obtain the target background image. Determine the target special effects video based on the target foreground image and the target background image, and display the target special effects video. The target special effects video includes the target foreground image of the image to be processed. The target foreground image includes a target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0076] The video background information can be content displayed in a layer below the target foreground image in the video frame, or content in the video frame other than the target foreground image. The video background image can be understood as the image behind the target foreground object in the video frame. The target background image can include an image that is the background information of the image to be processed. The background information of the image to be processed can be understood as image information in the image to be processed other than the target foreground object. The preset background image can be a pre-set template background image. Optionally, the preset background image includes solid color background images (such as black background images, white background images, and yellow background images), background images including preset patterns (such as stripes, grids, and irregular patterns), and background images of other images. The background extraction model can be a neural network model that takes an image as input to extract the background image of the image. The background extraction model can be used to extract the background from the input image. The method for determining the target foreground image can be found in the implementation methods described in the foregoing embodiments, and will not be repeated here.

[0077] In this embodiment of the disclosure, the background extraction model is obtained by training a second deep learning model based on a second sample image and a desired background image. Before applying the background extraction model provided in this embodiment, the pre-built second deep learning model can be trained in a supervised or unsupervised manner. Before training the model, multiple training samples can be constructed to train the model based on these multiple training samples. Optionally, the training process of the background extraction model can be as follows: acquiring multiple training samples; wherein the training samples include a second sample image and a desired background image; for each training sample, inputting the second sample image from the training sample into the pre-built second deep learning model to extract the background image from the second sample image based on the second deep learning model, and outputting the actual background image; determining a loss value based on the actual background image and the desired background image in the training samples; correcting the model parameters in the second deep learning model based on the loss value, and using the convergence of the loss function in the second deep learning model as the training objective, so as to use the trained second deep learning model as the background extraction model. The second sample image can be an image including a sample foreground object.

[0078] In this embodiment of the disclosure, after obtaining the image to be processed, the image can be input into a pre-trained background extraction model. Then, based on the background extraction model, background image extraction is performed on the image to be processed, and a target background image is output. Furthermore, a target special effects video can be determined based on the target background image and the target foreground image, and the target special effects video can be displayed.

[0079] It should be noted that there are several ways to determine the target special effects video based on the target foreground image and the target background image. Optionally, the target foreground image and / or the target background image are displayed according to a preset display duration and / or display format, and the resulting video is used as the target special effects video; or, at least one foreground generated image is generated based on the target foreground image, and the target foreground image, at least one foreground generated image, and the target background image are displayed according to a preset display duration and / or display format to obtain the target special effects video; or, the target special effects video is determined based on the target foreground image, the animation data corresponding to the target foreground image, the target background image, and the animation data corresponding to the target background image.

[0080] Optionally, the target special effects video includes a foreground animation and a background animation. Determining the target special effects video based on the target foreground image and the target background image includes: determining first animation data corresponding to the target foreground image; determining the foreground animation of the target special effects video based on the target foreground image and the first animation data; determining first animation data corresponding to the target background image; and determining the background animation of the target special effects video based on the target background image and the first animation data. The advantage of this setup is that it allows both the target foreground image and the target background image to be presented with animation effects in the target special effects video, enriching the special effects display effect of the target special effects video. Furthermore, it achieves the effect of separately processing the foreground image and the background image with animation effects.

[0081] The second animation data can be understood as data that causes the applied image and / or video to be presented with a preset animation effect. In this embodiment of the disclosure, the second animation data may be animation data corresponding to the target background image, that is, the second animation data is predetermined animation data applied to the target background image. Video background animation can be understood as animated elements and / or dynamic effects of the background portion of the video frame. For example, the second animation data may be a combination of border and camera movement animation data. Furthermore, the video background animation may be a frame with a border, the interior of which is the target background image, and the frame presents a camera movement animation effect.

[0082] In the embodiments of this disclosure, the method for determining the second animation data corresponding to the target background image can include various methods. Optionally, it can be determined from at least one preset animation data based on the image type identifier corresponding to the target background image; or, the preset animation data corresponding to the background image can be directly used as the second animation data, etc.

[0083] As an optional implementation of this embodiment, multiple foreground image types can be determined based on the object type of the foreground object, and animation data corresponding to each foreground image type can be determined respectively. The determined animation data is then used as candidate animation data corresponding to the foreground image. Similarly, multiple background image types can be determined, and animation data corresponding to each background image type can be determined respectively. The determined animation data is then used as candidate animation data corresponding to the background image. Further, after obtaining the target foreground image, a foreground image type identifier corresponding to the target foreground image can be determined. Further, based on the foreground image type identifier, animation data corresponding to the foreground image type identifier can be determined from a preset list of candidate animation data corresponding to the foreground image, and this animation data is used as first animation data corresponding to the target foreground image. Further, the target foreground image can be processed based on the first animation data to obtain the target foreground animation. And, after obtaining the target background image, a background image type identifier corresponding to the target background image can be determined. Further, based on the background image type identifier, animation data corresponding to the background image identifier can be determined from a preset list of candidate animation data corresponding to the background image, and this animation data is used as second animation data corresponding to the target background image. Furthermore, the target background image can be processed based on the second animation data to obtain the target background animation. Furthermore, the target foreground animation and the target background animation can be used as a target special effects video.

[0084] The technical solution of this disclosure embodiment obtains a target background image by inputting the image to be processed into a background extraction model, and determines the target special effects video based on the target foreground image and the target background image. This achieves the effect of extracting the background image of the image based on the background extraction model and determining the target special effects video based on the background image and the foreground image. It also enhances the connection between the target special effects video and the image to be processed, making the final target special effects video closer to the image to be processed.

[0085] Figure 4This is a flowchart illustrating another special effects processing method provided in this embodiment. Based on the above embodiments, the technical solution of this embodiment, after displaying the target special effects video, can further edit the target special effects video and update the target special effects video according to the special effects information editing operation. Optionally, after displaying the target special effects video, it further includes: responding to a special effects editing trigger operation for the target special effects video, displaying special effects editing items corresponding to the target special effects video; wherein, the special effects editing items include image content editing items and / or image effect editing items; receiving special effects information editing operations for the target special effects video through the special effects editing items, and updating the target special effects video according to the special effects information editing operation. For detailed implementation methods, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 4 As shown, the method in this embodiment may specifically include:

[0086] S410. In response to the special effects processing request, obtain the image to be processed, wherein the image to be processed includes the target foreground object.

[0087] S420. Perform special effects processing on the image to be processed to obtain a target special effects video, and display the target special effects video, wherein the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes a target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0088] S430, In response to an effect editing trigger operation targeting the effect video, display the effect editing items corresponding to the target effect video.

[0089] The special effects editing trigger operation can be understood as making the target special effects video editable, allowing for the editing of the special effects information corresponding to the target special effects video. Optionally, the special effects editing trigger operation includes at least one of the following operations: triggering a preset special effects editing control, inputting a click operation on the target special effects video (such as a single click, multiple clicks, etc.), receiving audio information including a trigger word associated with the special effects editing operation; receiving video editing instructions, etc. The special effects editing item can be understood as an interactive operation entry point for editing the special effects information corresponding to the target special effects video. The special effects editing item can be used to edit the special effects information corresponding to the target special effects video. Optionally, the special effects editing item includes an image content editing item and / or an image effect editing item. The image content editing item can be used to edit the image content included in the target special effects video. In this embodiment of the disclosure, the image content included in the target special effects video includes at least one of an image to be processed, a target foreground image, and a target background image. The image content editing item includes at least one of a first image editing item corresponding to the image to be processed, a second image editing item corresponding to the target foreground image, and a third image editing item corresponding to the target background image. The first image editing option allows you to edit the content of the image to be processed. The second image editing option allows you to edit the content of the target foreground image. The third image editing option allows you to edit the content of the target background image. The image effects editing option allows you to edit the image effects presented in the target special effects video. The image effects editing option may include animation effects editing options and / or display duration editing options. The animation effects editing option allows you to edit the animation effects presented in the target special effects video. The display duration editing option allows you to edit the display duration of the target special effects video.

[0090] As an optional implementation of this disclosure, after displaying the target special effects video, the display interface of the target special effects video may include a special effects editing control. Furthermore, upon detecting a control trigger operation on the special effects editing control, it can be determined that a special effects editing trigger operation for the target special effects video has been received. Furthermore, in response to the special effects editing trigger operation, special effects editing items corresponding to the target special effects video are displayed.

[0091] S440: Receive special effects information editing operations for the target special effects video through the special effects editing item, and update the target special effects video according to the special effects information editing operations.

[0092] The special effects information editing operation can be understood as the operation of editing the special effects information corresponding to the target special effects video. Optionally, the special effects information editing operation corresponding to the image content editing item includes at least one of the following operations: image replacement, image addition, image display order adjustment, and image deletion. The special effects information editing operation corresponding to the animation effect editing item includes at least one of the following operations: animation effect replacement, animation effect addition, and animation effect deletion.

[0093] In this embodiment of the disclosure, when displaying the special effects editing item corresponding to the target special effects video, the user can input special effects information editing operations for the target special effects video through the special effects editing item. Furthermore, upon receiving a special effects information editing operation for the target special effects video, the target special effects video can be updated based on the received special effects editing operation.

[0094] As an optional implementation of this embodiment, when the special effects information editing operation includes an image display order adjustment operation, updating the target special effects video according to the special effects information editing operation includes: displaying video images included in the target special effects video according to a preset image display method; wherein, the image display method includes image display layers and / or image display order; the video images include a target foreground image or the video images include a target foreground image and a video background image; receiving an image display order adjustment operation input for the video images, updating the image display method of the video images according to the image display order adjustment operation, and changing the image display order of the video images in the target special effects video, and previewing the updated target special effects video.

[0095] More specifically, upon receiving an effects information editing operation, the video images included in the target effects video can be identified, and displayed according to the image display layer and image display order. This ensures that the target foreground image in the video images is displayed in the foreground layer display area, and the video background image in the video images is displayed in the background layer display area. To facilitate targeted interaction with each video image, multiple slots can be set in the foreground layer display area and / or the background layer display area, each slot being used to hold a single video image. The target foreground image and / or video background image are displayed in the slots according to the image display order of the video images in the target effects video. Furthermore, upon detecting an image display order adjustment operation for the target foreground image, the slot occupied by the target foreground image in the foreground layer display area can be updated according to the image display order adjustment operation. If a video background image associated with the target foreground image exists, the display position of the video background image in the background layer display area can be updated according to the updated target foreground image. Furthermore, the image display order of the video image in the target effects video can be updated according to the updated video image. Here, a slot can be understood as a data empty space that can be used to add video images.

[0096] As an optional implementation of this disclosure, when the special effects information editing operation includes an image replacement operation, updating the target special effects video according to the special effects information editing operation includes: displaying video images included in the target special effects video according to a preset image display method; wherein, the image display method includes image display layers and image display order; the video image includes a target foreground image or the video image includes a target foreground image and a video background image; when receiving an image replacement operation for the video background image, determining the replaced video background image according to the image replacement operation, and updating the target special effects video according to the replaced video background image and its associated target foreground image.

[0097] More specifically, upon receiving an effects information editing operation, the video images included in the target effects video can be displayed according to the image display layer and image display order. These video images may include a target foreground image or a video image that includes both a target foreground image and a video background image. Further, upon receiving an image replacement operation for the video background image, at least one candidate effects image can be displayed, which may include a candidate background image and / or a candidate image. Subsequently, upon receiving an image selection operation for the candidate image, a background image corresponding to that candidate image can be determined, and the video background image can be replaced based on that background image. Further, the target effects video can be updated based on the replaced video background image and the target foreground image.

[0098] As an optional implementation of this disclosure, when the special effects information editing operation includes an image deletion operation, updating the target special effects video according to the special effects information editing operation includes: displaying video images included in the target special effects video according to a preset image display mode; wherein, the image display mode includes image display layers and image display order; the video images include a target foreground image or the video images include a target foreground image and a video background image; receiving an image deletion operation for the target foreground image and / or the video background image, and if there is an associated image related to the image to be deleted, deleting the image to be deleted from its slot to update the target special effects video, wherein the display mode of the remaining video images remains unchanged; and, if there is no associated image related to the image to be deleted, deleting the image to be deleted from its slot, and making the remaining video images play continuously by sequentially shifting them forward according to their relative display relationship in the target video file; and updating the target special effects video according to the adjusted video images. Specifically, when the image to be deleted is the target foreground image, the associated image is the video background image; when the image to be deleted is the video background image, the associated image is the target foreground image. The relative display relationship includes the associated display method of the video foreground image and the video background image, such as relative display time.

[0099] Optionally, the image effects editing items include animation effects editing items; the animation effects editing items include animation duration editing items; receiving special effects information editing operations for the target special effects video through the special effects editing items, and updating the target special effects video according to the special effects information editing operations, including: receiving animation duration editing operations for the target special effects video through the animation duration editing items, determining the animation target and animation duration editing information in the target special effects video according to the animation duration editing operations; determining the animation duration based on the object display duration of the animation target and the animation duration editing information, and updating the target special effects video according to the animation target and the animation duration. The advantage of this setting is that it enables the customization and editing of the animation duration in the target special effects video after obtaining the target special effects video, thereby enriching the special effects processing methods and the special effects display effects of the special effects video.

[0100] The animation duration editing item can be understood as an interactive operation entry point for editing the animation display duration of the target special effects video. The display form of the animation duration editing item can include various options, including a duration pointer slider and / or a duration editing box. The animation duration editing operation can be understood as an operation for editing the animation display video of the target special effects video. The animation duration editing operation can include the selection operation of the animation target and the editing operation of duration information. The animation target can be understood as the target object of the relevant animation data. In other words, when determining the target special effects video, first animation data corresponding to the target foreground image and / or second animation data corresponding to the target background image can be determined. Further, the first animation data can be applied to the target foreground image to obtain the video foreground animation; and the second animation data can be applied to the target background image to obtain the video background animation. The target foreground image applied by the first animation data and the target background image applied by the second animation data can be used as the animation target. Optionally, the animation target includes at least one of the target foreground image, the target foreground object in the target foreground image, and the target background image. Animation duration editing information can be understood as the duration information obtained after editing the animation duration. Object display duration can be the display duration of the corresponding animated object in the target special effects video. Animation effect duration can be understood as the duration of the animation data's effect; or, animation effect duration can also be understood as the display duration of the animation effect.

[0101] As an optional implementation of this embodiment, when the displayed special effects editing items include an animation duration editing item, the animation duration editing item can be displayed in the editing interface corresponding to the target special effects video in the form of a duration pointer slider. Further, when an object selection operation for a candidate animation target is detected, the animation target in the target special effects video can be determined based on the object selection operation, and the duration pointer slider corresponding to the animation target can be displayed. Further, when a pointer editing operation is detected for the duration pointer slider, animation duration editing information can be determined based on the pointer editing operation. Further, when the animation target and animation duration editing information are determined, the object display duration of the animation target in the target special effects video can be determined, and the object display duration can be processed according to the animation duration editing information to determine the animation duration of the animation target. Further, the target special effects video can be updated based on the animation target and animation duration. For example, the animation duration of the target special effects video can be represented by a progress value of 0-100%, with 50% as the duration center point. Furthermore, the display position of the duration center point in the duration pointer slider can be adjusted, and the animation duration scaling ratio can be determined based on the adjusted display position. The animation duration can then be determined based on this scaling ratio, and the target special effects video can be updated based on the animation duration.

[0102] For example, Figure 5 This is a schematic diagram illustrating an interface display of the special effects editing process provided in an embodiment of this disclosure. Figure 5 As shown, the display interface includes a video display area 51 and a video editing area 52. The video display area 51 can be used to display the target special effects video. The video editing area 52 can be used to display the animation objects included in the target special effects video and the special effects editing items corresponding to the animation objects. The video editing area 52 displays the animation objects in the form of slots, including background slots and foreground slots. The background slots include background animation 1, background animation 2, and background animation 3. The foreground slots include foreground animation 1, foreground animation 2, and foreground animation 3. When a drag operation is detected on the animation identifier of background animation 1, the slot position of background animation 1 can be changed according to the animation drag operation to change its display order in the target special effects video. Furthermore, when it is detected that background animation 1 is dragged to the slot position of background animation 2 based on the animation drag operation, the slot position of background animation 2 can be swapped with the slot position of background animation 1. Upon detecting a selection trigger operation for background animation 1, special effects editing items corresponding to background animation 1 can be displayed based on the video editing area 52. These special effects editing items may include special effects editing item A, special effects editing item B, and special effects editing item C. Special effects editing items A, B, and C are used to edit different special effects information. Furthermore, upon detecting a special effects information editing operation for special effects editing item A, B, and / or C, special effects editing information can be determined based on the special effects information editing operation. Then, the target special effects video is updated based on the special effects editing information.

[0103] The technical solution of this disclosure, in response to a special effects editing trigger operation for a target special effects video, displays special effects editing items corresponding to the target special effects video; wherein, the special effects editing items include image content editing items and / or image effect editing items; furthermore, by receiving special effects information editing operations for the target special effects video through the special effects editing items, and updating the target special effects video according to the special effects information editing operations, the effect of custom editing of the special effects information of the target special effects video is realized when the target special effects video is obtained, which further enriches the special effects processing methods, enriches the special effects display effect of the special effects video, and improves the special effects processing experience.

[0104] Figure 6 This is a flowchart illustrating a special effects processing method provided in an embodiment of this disclosure. This embodiment is a preferred embodiment of the above embodiments. Figure 6 As shown, the method in this embodiment may specifically include:

[0105] First, the image to be processed is acquired, which includes the target foreground object and the corresponding background information. Further, the background region of the image to be processed can be extracted, and the region within the background corresponding to the target foreground object can be filled to obtain the target background image. Then, combined animation data with border effects and camera movement effects can be applied to the target background image to obtain a video background animation. Additionally, the target foreground object of the image to be processed can be extracted using a stylization model corresponding to the plush style, and the target foreground object can be stylized to obtain a target foreground image. Then, combined animation data including appearance animation and camera movement can be applied to the target foreground image to obtain a video foreground animation. Furthermore, the video background animation and video foreground animation can be used as the target special effects video.

[0106] The technical solution of this disclosure can automatically identify and extract the target foreground object in the image to be processed during the special effects processing, and perform stylization processing on the target foreground object to obtain a stylized target foreground image. Then, a target special effects video containing the target foreground image is generated. This realizes differentiated processing of the target foreground object in the image to be processed, enriches the special effects processing methods, increases the content hierarchy of the image after special effects processing, and enhances the interactive experience of special effects.

[0107] Figure 7 This is a schematic diagram of the structure of a special effects processing device provided in an embodiment of the present disclosure, as shown below. Figure 7 As shown, the device includes: a special effects processing request module 510 and a special effects video display module 520. The special effects processing request module 510 is used to acquire an image to be processed in response to a special effects processing request, wherein the image to be processed includes a target foreground object. The special effects video display module 520 is used to perform special effects processing on the image to be processed to obtain a target special effects video and to display the target special effects video; wherein the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object, and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0108] The technical solution of this embodiment responds to a special effects processing request by a special effects processing request module 510 to acquire an image to be processed. The acquisition of the image to be processed can be triggered through a simple interactive operation. Furthermore, the image to be processed includes a target foreground object, providing a data foundation for subsequent special effects processing of the foreground object in the image. Further, a special effects video display module 520 performs special effects processing on the image to be processed to obtain a target special effects video, which is then displayed. The target special effects video includes a target foreground image of the image to be processed. The target foreground image includes a target foreground object, and at least a portion of the target foreground object's object area is presented in a preset style. This solves the problem of monotonous and uninteresting presentation of image special effects in related technologies. During the special effects processing, the target foreground object in the image to be processed can be automatically identified and extracted, and stylized to obtain a stylized target foreground image. This, in turn, generates a target special effects video containing the target foreground image, achieving differentiated processing of the target foreground object in the image to be processed, enriching the special effects processing methods, increasing the content hierarchy of the processed image, and enhancing the interactive experience of the special effects.

[0109] Based on any optional technical solution in the embodiments of this disclosure, optionally, the target foreground image is presented as video foreground information in the target special effects video; the special effects video display module 520 includes: a special effects video first determining unit. The special effects video first determining unit is used to input the image to be processed into a stylization processing model to obtain a target foreground image, and to determine a target special effects video based on the target foreground image; wherein the stylization processing model is obtained by training a first deep learning model based on a first sample image and a desired foreground image; the desired foreground image includes sample foreground objects in the first sample image, and the sample foreground objects are presented in a preset style.

[0110] Based on any optional technical solution in the embodiments of this disclosure, optionally, the special effects video first determining unit is specifically used to determine the first animation data corresponding to the target foreground image, determine the video foreground animation based on the target foreground image and the first animation data, and determine the target special effects video based on the video foreground animation.

[0111] Based on any optional technical solution in the embodiments of this disclosure, optionally, the target foreground image is presented as video foreground information in the target special effects video; the video background information of the target special effects video presents a video background image; the video background image includes at least one of the image to be processed, the target background image of the image to be processed, and a preset background image.

[0112] Based on any optional technical solution in the embodiments of this disclosure, optionally, the video background image includes the target background image of the image to be processed; the special effects video display module 520 includes: a special effects video second determining unit. The special effects video second determining unit is used to determine the target foreground image of the image to be processed, and to input the image to be processed into a background extraction model to obtain the target background image, and to determine a target special effects video based on the target foreground image and the target background image; wherein the background extraction model is obtained by training a second deep learning model based on a second sample image and a desired background image.

[0113] Based on any optional technical solution in the embodiments of this disclosure, optionally, the target special effects video includes a video foreground animation and a video background animation; the second special effects video determining unit includes: a foreground animation determining subunit and a background animation determining subunit. The foreground animation determining subunit is used to determine first animation data corresponding to the target foreground image, and to determine the video foreground animation of the target special effects video based on the target foreground image and the first animation data; the background animation determining subunit is used to determine second animation data corresponding to the target background image, and to determine the video background animation of the target special effects video based on the target background image and the second animation data.

[0114] Based on any optional technical solution in the embodiments of this disclosure, optionally, in the target special effects video, the target foreground image is displayed with a first animation effect; and, when the target special effects video includes the target background image of the image to be processed, the target foreground image is displayed with a first animation effect and the target background image is displayed with a second animation effect; and, when the target special effects video includes the image to be processed and the target background image of the image to be processed, the target foreground image is displayed with a first animation effect, the target background image is displayed with a second animation effect, and the image to be processed is displayed with a third animation effect.

[0115] Based on any optional technical solution in the embodiments of this disclosure, the special effects video display module 520 is specifically used to determine the image special effects video corresponding to each of the images to be processed when there are multiple images to be processed, and to determine the target special effects video based on the image special effects videos corresponding to the multiple images to be processed.

[0116] Based on any optional technical solution in the embodiments of this disclosure, the device may optionally further include: an editing item display module and a special effects editing module. The editing item display module is configured to, after displaying the target special effects video, in response to a special effects editing trigger operation for the target special effects video, display special effects editing items corresponding to the target special effects video; wherein the special effects editing items include image content editing items and / or image effect editing items; the special effects editing module is configured to receive special effects information editing operations for the target special effects video through the special effects editing items, and update the target special effects video according to the special effects information editing operations.

[0117] Based on any optional technical solution in the embodiments of this disclosure, optionally, the image effect editing item includes an animation effect editing item; the animation effect editing item includes an animation duration editing item; the special effects editing module includes: a duration editing unit and a special effects video updating unit. The duration editing unit is used to receive an animation duration editing operation for the target special effects video through the animation duration editing item, and determine the animation target and animation duration editing information in the target special effects video based on the animation duration editing operation; the special effects video updating unit is used to determine the animation duration based on the object display duration of the animation target and the animation duration editing information, and update the target special effects video based on the animation target and the animation duration.

[0118] The special effects processing apparatus provided in this disclosure can execute the special effects processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the special effects processing method.

[0119] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0120] The following is for reference. Figure 8 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0121] like Figure 8 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0122] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0123] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0124] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0125] The electronic device provided in this disclosure and the special effects processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this disclosure can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0126] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the special effects processing method provided in the above embodiments.

[0127] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0128] According to one or more embodiments of this disclosure, [Example 1] provides a special effects processing method, including: in response to a special effects processing request, acquiring an image to be processed, wherein the image to be processed includes a target foreground object; performing special effects processing on the image to be processed to obtain a target special effects video, and displaying the target special effects video; wherein the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0129] According to one or more embodiments of this disclosure, [Example 2] provides the method of Example 1, further comprising: optionally, the target foreground image is presented as video foreground information in the target special effects video; the step of performing special effects processing on the image to be processed to obtain the target special effects video includes: inputting the image to be processed into a stylization processing model to obtain a target foreground image, and determining the target special effects video based on the target foreground image; wherein the stylization processing model is obtained by training a first deep learning model based on a first sample image and a desired foreground image; the desired foreground image includes sample foreground objects in the first sample image and the sample foreground objects are presented in a preset style.

[0130] According to one or more embodiments of this disclosure, [Example 3] provides the method of Example 2, which further includes: optionally, determining a target special effects video based on the target foreground image, including: determining first animation data corresponding to the target foreground image, determining a video foreground animation based on the target foreground image and the first animation data, and determining the target special effects video based on the video foreground animation.

[0131] According to one or more embodiments of this disclosure, Example 4 provides the method of Example 1, which further includes: optionally, the target foreground image is presented as video foreground information in the target special effects video; the video background information of the target special effects video presents a video background image; the video background image includes at least one of the image to be processed, the target background image of the image to be processed, and a preset background image.

[0132] According to one or more embodiments of this disclosure, Example 5 provides the method of Example 4, further comprising: optionally, the video background image includes a target background image of the image to be processed; the step of performing special effects processing on the image to be processed to obtain a target special effects video includes: determining a target foreground image of the image to be processed, and inputting the image to be processed into a background extraction model to obtain a target background image, and determining a target special effects video based on the target foreground image and the target background image; wherein the background extraction model is obtained by training a second deep learning model based on a second sample image and a desired background image.

[0133] According to one or more embodiments of this disclosure, Example Six provides the method of Example Five, which further includes: optionally, the target special effects video includes a video foreground animation and a video background animation; the step of determining the target special effects video based on the target foreground image and the target background image includes: determining first animation data corresponding to the target foreground image, determining the video foreground animation of the target special effects video based on the target foreground image and the first animation data; determining second animation data corresponding to the target background image, and determining the video background animation of the target special effects video based on the target background image and the second animation data.

[0134] According to one or more embodiments of this disclosure, Example 7 provides the method of Example 1, further comprising: optionally, in the target special effects video, the target foreground image is displayed with a first animation effect; and, when the target special effects video includes a target background image of the image to be processed, the target foreground image is displayed with a first animation effect and the target background image is displayed with a second animation effect; and, when the target special effects video includes the image to be processed and the target background image of the image to be processed, the target foreground image is displayed with a first animation effect, the target background image is displayed with a second animation effect, and the image to be processed is displayed with a third animation effect.

[0135] According to one or more embodiments of this disclosure, Example 8 provides the method of Example 1, which further includes: optionally, performing special effects processing on the image to be processed to obtain a target special effects video, including: when there are multiple images to be processed, determining an image special effects video corresponding to each image to be processed, and determining the target special effects video based on the image special effects videos corresponding to the multiple images to be processed.

[0136] According to one or more embodiments of this disclosure, Example Nine provides the method of Example One, which further includes: optionally, after displaying the target special effects video, the method further includes: in response to a special effects editing trigger operation for the target special effects video, displaying special effects editing items corresponding to the target special effects video; wherein the special effects editing items include image content editing items and / or image effect editing items; receiving special effects information editing operations for the target special effects video through the special effects editing items, and updating the target special effects video according to the special effects information editing operations.

[0137] According to one or more embodiments of this disclosure, Example 10 provides the method of Example 9, which further includes: optionally, the image effect editing item includes an animation effect editing item; the animation effect editing item includes an animation duration editing item; the step of receiving an effect information editing operation for the target effect video through the effect editing item, and updating the target effect video according to the effect information editing operation, includes: receiving an animation duration editing operation for the target effect video through the animation duration editing item, determining the animation target and animation duration editing information in the target effect video according to the animation duration editing operation; determining the animation duration according to the object display duration of the animation target and the animation duration editing information, and updating the target effect video according to the animation target and the animation duration.

[0138] According to one or more embodiments of this disclosure, [Example 11] provides a special effects processing apparatus, comprising: a special effects processing request module, configured to acquire an image to be processed in response to a special effects processing request, wherein the image to be processed includes a target foreground object; and a special effects video display module, configured to perform special effects processing on the image to be processed to obtain a target special effects video, and display the target special effects video; wherein the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0139] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0140] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0141] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to a special effects processing request, acquire an image to be processed, wherein the image to be processed includes a target foreground object; perform special effects processing on the image to be processed to obtain a target special effects video, and display the target special effects video; wherein the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

[0142] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0144] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not necessarily limiting in certain circumstances; for example, a special effects processing module can also be described as a "module for acquiring an image to be processed".

[0145] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0146] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0147] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0148] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0149] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A special effects processing method, characterized in that, include: In response to a special effects processing request, an image to be processed is obtained, wherein the image to be processed includes a target foreground object; The image to be processed is subjected to special effects processing to obtain a target special effects video, and the target special effects video is displayed; wherein, the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

2. The special effects processing method according to claim 1, characterized in that, The target foreground image is presented as video foreground information in the target special effects video; the process of performing special effects processing on the image to be processed to obtain the target special effects video includes: The image to be processed is input into a stylization processing model to obtain a target foreground image, and a target special effects video is determined based on the target foreground image; wherein, the stylization processing model is obtained by training a first deep learning model based on a first sample image and a desired foreground image; the desired foreground image includes sample foreground objects in the first sample image and the sample foreground objects are presented in a preset style.

3. The special effects processing method according to claim 2, characterized in that, The step of determining the target special effects video based on the target foreground image includes: First animation data corresponding to the target foreground image is determined, video foreground animation is determined based on the target foreground image and the first animation data, and target special effects video is determined based on the video foreground animation.

4. The special effects processing method according to claim 1, characterized in that, The target foreground image is presented as video foreground information in the target special effects video; the video background information of the target special effects video includes a video background image. The video background image includes at least one of the image to be processed, the target background image of the image to be processed, and a preset background image.

5. The special effects processing method according to claim 4, characterized in that, The video background image includes the target background image of the image to be processed; The process of performing special effects processing on the image to be processed to obtain the target special effects video includes: The process involves determining the target foreground image of the image to be processed, inputting the image to be processed into a background extraction model to obtain a target background image, and determining a target special effects video based on the target foreground image and the target background image; wherein the background extraction model is obtained by training a second deep learning model based on a second sample image and a desired background image.

6. The special effects processing method according to claim 5, characterized in that, The target special effects video includes foreground animation and background animation; The step of determining the target special effects video based on the target foreground image and the target background image includes: Determine the first animation data corresponding to the target foreground image, and determine the video foreground animation of the target special effects video based on the target foreground image and the first animation data; Determine the second animation data corresponding to the target background image, and determine the video background animation of the target special effects video based on the target background image and the second animation data.

7. The special effects processing method according to claim 1, characterized in that, In the target special effects video, the target foreground image is displayed with a first animation effect; and, when the target special effects video includes a target background image of the image to be processed, the target foreground image is displayed with a first animation effect and the target background image is displayed with a second animation effect; and, when the target special effects video includes the image to be processed and the target background image of the image to be processed, the target foreground image is displayed with a first animation effect, the target background image is displayed with a second animation effect, and the image to be processed is displayed with a third animation effect.

8. The special effects processing method according to claim 1, characterized in that, The process of performing special effects processing on the image to be processed to obtain the target special effects video includes: In the case of multiple images to be processed, an image effect video corresponding to each image to be processed is determined, and a target effect video is determined based on the image effect videos corresponding to the multiple images to be processed.

9. The special effects processing method according to claim 1, characterized in that, Following the display of the target special effects video, the following is also included: In response to a special effects editing trigger operation for the target special effects video, special effects editing items corresponding to the target special effects video are displayed; wherein, the special effects editing items include image content editing items and / or image effect editing items; The special effects editing item receives special effects information editing operations for the target special effects video, and updates the target special effects video according to the special effects information editing operations.

10. The special effects processing method according to claim 9, characterized in that, The image effect editing items include animation effect editing items; the animation effect editing items include animation duration editing items; the step of receiving special effects information editing operations for the target special effects video through the special effects editing items, and updating the target special effects video according to the special effects information editing operations, includes: The animation duration editing item receives the animation duration editing operation for the target special effects video, and determines the animation target and animation duration editing information in the target special effects video based on the animation duration editing operation; The animation duration is determined based on the object display duration of the animation target and the animation duration editing information, and the target special effects video is updated based on the animation target and the animation duration.

11. A special effects processing device, characterized in that, include: The special effects processing request module is used to respond to a special effects processing request and obtain an image to be processed, wherein the image to be processed includes a target foreground object; A special effects video display module is used to perform special effects processing on the image to be processed to obtain a target special effects video and display the target special effects video; wherein, the target special effects video includes a target foreground image of the image to be processed; the target foreground image includes the target foreground object and at least a portion of the object area of ​​the target foreground object is presented in a preset style.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects processing method as described in any one of claims 1-10.

13. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the special effects processing method as described in any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the special effects processing method as described in any one of claims 1-10.

Images