Methods, apparatus, computer equipment, and storage media for generating special effects

Abstract

This disclosure provides a method, apparatus, computer equipment, and storage medium for generating special effects, the method comprising the steps of: acquiring selected special effect material in response to a selection operation by a user; displaying a preview effect of the special effect, including a face image and special effect material, on a special effect editing page; determining a target area image corresponding to the target area from a face image in response to a selection operation for a target area, and updating the preview effect of the special effect based on the target area image; and generating a target special effect in response to a decision operation on the updated preview effect of the special effect, based on the display information of the target area and special effect material in the updated preview effect of the special effect. This enhances the method for generating special effects and improves the diversity and quality of the generated special effects.

Description

Technical Field

[0004] , , , , , , , , , ,

[0005]

[0001] This application claims the priority of Chinese Patent Application No. 202310484117.0 filed on April 28, 2023, and the entire content disclosed in the above Chinese Patent Application is incorporated herein by reference.

[0002] This disclosure relates to a method, apparatus, computer device, and storage medium for generating special effects.

Background Art

[0003] With the development of short video technology, the number of users who use short videos to obtain information and share their lives has increased. When users shoot short videos, it is often necessary to use special effects to beautify the videos. In actual applications, different special effects may have different presentation forms, and most of the video playback effects depend on the selection of special effects, which poses a strict test for the production of special effects.

Summary of the Invention

Means for Solving the Problems

[0004] Embodiments of this disclosure provide at least a method, apparatus, computer device, and storage medium for generating special effects.

[0005] According to a first aspect, embodiments of this disclosure provide a method for generating special effects, responding to a selection operation by a user to obtain a selected special effect material; displaying a preview effect of a special effect including a face image and the special effect material on a special effect editing page; responding to a selection operation on a target part, determining a target part image corresponding to the target part from the face image, and updating the preview effect of the special effect based on the target part image; The process includes the step of generating a target special effect in response to a decision operation on the preview effect of the updated special effect, based on the display information of the target portion within the preview effect of the updated special effect and the special effect material.

[0006] In one possible embodiment, if the selected special effect material is a background image, the method is: How to collect video with a camera, A method for obtaining a preview effect of a special effect includes identifying contour information within video frames of the video, separating a human body region image from the video frames based on the contour information, and superimposing the human body region image onto a background image. The method further includes generating a preview effect of a special effect based on a method for obtaining a preview effect of a special effect.

[0007] In one possible embodiment, if the selected special effect material is a background image, the method is: A method for determining the display information of the background image based on the target position of the target object in the background image and the size information of the background image, when it is detected that the background image contains a target object, A method for displaying the background image according to the display information, and further including the step of displaying the background image in a preview effect of the special effect based on the method for displaying the background image in the preview effect of the special effect.

[0008] In one possible embodiment, if the special effect material is a keypoint control material, the face image becomes a reference image, and the preview effect of the special effect is displayed at a first position. The method described above The step further includes displaying a real-time preview effect of the special effect, including the special effect material and the face image, at a second location on the special effect editing page. After the step of responding to a selection operation on a target site, the method A step of displaying a preview effect of the special effect at the second position and a real-time preview effect of the special effect at the first position, further comprising the step of the display area at the first position being larger than the display area at the second position.

[0009] In one possible embodiment, the step in response to a selection operation on a target site is: A step that responds to a selection operation performed on a target area of ​​a face image within the preview effect of the special effect, or The step includes responding to a trigger operation on a target button corresponding to the target area.

[0010] In one possible embodiment, the display information of the target area includes the relative positional relationship between the target area image and the special effect material. The method further includes the step of determining the relative positional relationship between the target area image and the special effect material based on the following method: The first position information of the target area image set in the preview effect of the updated special effect is determined, and the second position information of the special effect material set in the preview effect of the updated special effect is determined. Based on the first and second position information, the relative positional relationship between the target area image and the special effect material is determined.

[0011] In one possible embodiment, after the step of updating the preview effect of the special effect based on the target area image, the method, The process further includes the step of determining the display information of the target area after editing in response to an editing operation on the target area image, and adjusting the display format of the target area according to the display information.

[0012] In one possible embodiment, the step of obtaining a preview effect of the special effect is: The steps include rendering the background image based on the background orthogonal camera processing module to obtain a background texture, The steps include inputting the background texture and the video to the main perspective camera processing module, The main perspective camera processing module includes the step of replacing other textures other than the contour texture corresponding to the contour information based on the background texture, rendering, and obtaining a preview effect of the special effect.

[0013] In one possible embodiment, the step of updating the preview effect of the special effect based on the target area image is: The steps include rendering the target area image based on the foreground orthogonal camera processing module and obtaining the target area texture, The process includes the step of overlaying the target area texture onto the currently displayed screen texture of the preview effect of the special effect.

[0014] According to a second aspect, the present embodiment further provides a special effect generating device, A module for acquiring selected special effect materials in response to user selection operations, The special effects editing page includes a first display module for displaying a preview effect of a special effect, which includes a face image and the aforementioned special effects material, A second display module for responding to a selection operation on a target area, determining a target area image corresponding to the target area from the face image, and updating the preview effect of the special effect based on the target area image, The system includes a generation module for generating a target special effect in response to a decision operation on the preview effect of the updated special effect, based on the display information of the target portion within the preview effect of the updated special effect and the special effect material.

[0015] According to a third aspect, embodiments of the present disclosure further provide a computer device including a processor, a memory, and a bus, where device-readable instructions executable by the processor are stored in the memory, and when the computer device executes, the processor and the memory communicate via the bus, and when the device-readable instructions are executed, the processor is caused to execute the steps in the first aspect or any one of the possible embodiments of the first aspect.

[0016] According to a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium storing a computer program which, when executed, causes a processor to execute the steps in the first aspect or any one of the possible embodiments of the first aspect.

[0017] To make the above objects, features, and advantages of the present disclosure clearer and easier to understand, the following will be described in detail with reference to preferred embodiments and the accompanying drawings.

[0018] To more clearly explain the technical solutions of the embodiments of the present disclosure, the drawings necessary for use in the following embodiments will be briefly introduced. Here, the drawings are incorporated into the specification and form a part of this specification. These drawings show embodiments consistent with the present disclosure and are useful for explaining the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show specific embodiments of the present disclosure and are not considered to limit the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative labor.

Brief Description of the Drawings

[0019] [Figure 1] FIG. 1 shows a flowchart of a method for generating a special effect according to an embodiment of the present disclosure. [Figure 2a] FIG. 2a shows a schematic diagram of a special effect type selection page in a method for generating a special effect according to an embodiment of the present disclosure. [Figure 2b]Figure 2b shows a schematic diagram of the preview effect of a special effect in the method for generating special effects according to the embodiment of this disclosure. [Figure 2c] Figure 2c shows a schematic diagram of the display of a target area image in the method for generating special effects according to the embodiment of this disclosure. [Figure 3] Figure 3 shows a flowchart illustrating a method for determining the relative positional relationship between a target area image and a special effect material according to the embodiment of this disclosure. [Figure 4] Figure 4 shows a schematic diagram of the structure of a special effect generating device according to the embodiment of this disclosure. [Figure 5] Figure 5 shows a schematic diagram of the structure of a computer device according to an embodiment of this disclosure. [Modes for carrying out the invention]

[0020] To clarify the purpose, technical solutions, and advantages of the embodiments of this disclosure, the technical solutions of the embodiments of this disclosure will be described clearly and completely below with reference to the drawings of the embodiments. It is clear that the embodiments described are only some, not all, embodiments of the disclosure. Typically, the assemblies of the embodiments of this disclosure described and shown herein may be set up and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of this disclosure provided in the drawings is intended to represent only selected embodiments of the disclosure without limiting the scope of the claimed disclosure. All other embodiments obtained by a person skilled in the art based on the embodiments of this disclosure, without requiring any creative work, will also fall within the scope of protection of this disclosure.

[0021] Note that similar reference numbers and letters represent similar items in subsequent drawings. Therefore, if an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0022] The term "and / or" in this application is merely for the purpose of describing a related relationship, indicating that three types of relationships exist. For example, in the case of A and / or B, this refers to three situations: A existing alone, A and B existing together, and B existing alone. Furthermore, the term "at least one" in this application refers to any one of a plurality, or any combination of at least two of a plurality. For example, including at least one of A, B, and C means including any one or more elements selected from the set composed of A, B, and C.

[0023] To ensure understanding, before using any of the technical solutions disclosed in each embodiment of this disclosure, the user should be informed in an appropriate manner in accordance with applicable laws and regulations of the type, scope, and usage scenarios of the personal information related to this disclosure, and the user's consent should be obtained.

[0024] For example, by responding to the receipt of an active request from the user and sending prompt information to the user, the user is clearly prompted that the operation it requests to perform requires the acquisition and use of the user's personal information. This allows the user to freely choose whether or not to provide personal information to software or hardware such as electronic devices, application programs, servers, or storage media that perform the operation of the technical solution of this disclosure based on the prompt information.

[0025] A selectable but non-restrictive implementation method is to respond to the receipt of an active user request and send prompt information to the user, which may be, for example, a pop-up window, in which case the prompt information may be presented in text format. The pop-up window may also include selection controls for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0026] To be understood, the above notice and user consent process are merely illustrative and do not limit the methods of implementing this disclosure. Other methods that comply with applicable laws and regulations are also applicable to implementing this disclosure.

[0027] To facilitate understanding of this embodiment, the method for generating special effects disclosed in this embodiment will first be described in detail. The execution body of the method for generating special effects according to this embodiment is generally a computer device having a certain computing power, and this computer device includes, for example, terminal equipment, a server, or other processing equipment. Terminal equipment may be user equipment (UE), mobile devices, user terminals, terminals, cellular phones, mobile phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle equipment, wearable devices, etc. In some possible implementations, the method for generating special effects can be implemented by a processor calling computer-readable instructions stored in memory.

[0028] Figure 1 is a flowchart of a method for generating special effects according to an embodiment of the present disclosure, the method comprising steps 101 to 104, Step 101: In response to the user's selection operation, retrieve the selected special effect material.

[0029] Step 102: On the special effects editing page, display a preview of the special effect, including the face image and the aforementioned special effect materials.

[0030] Step 103: In response to a selection operation for a target area, a target area image corresponding to the target area is determined from the face image, and the preview effect of the special effect is updated based on the target area image.

[0031] Step 104: In response to a decision operation on the preview effect of the updated special effect, a target special effect is generated based on the display information of the target portion within the preview effect of the updated special effect and the special effect material.

[0032] The following is a detailed explanation of the above steps.

[0033] For step 101, the special effect material may be a component of the special effect, and in one possible embodiment, the selectable special effect materials corresponding to different types of special effects may be different.

[0034] Here, the user's selection operation includes a selection operation for selecting a special effect type and a selection operation for selecting a specific special effect material.

[0035] As an example, the selection interface is shown in Figure 2a. In Figure 2a, the Type 1 sticker, Type 2 sticker, Type 3 sticker, Type 4 sticker, etc., may all refer to different special effect types. After any special effect type is triggered, the user selects a special effect material to trigger it again, thereby enabling the selection of a special effect material.

[0036] Here, the special effect material selected by the user trigger may refer to any of the special effect materials provided by the system (i.e., the application program), or any media content may be selected from the client's local machine as the special effect material. For example, any drawing can be selected from the local machine as the special effect material.

[0037] Here, it should be noted that the special effects materials include, but are not limited to, static drawings, dynamic drawings, videos, and animations.

[0038] In one possible embodiment, if the special effects material is uploaded from a local source, the size of the special effects material (which may be the size of the memory it occupies and / or the resolution, etc.) affects the amount of memory occupied by the ultimately generated target special effects. Since the generated target special effects must be compatible with all devices, if the target special effects occupy too much memory on some poorly performing devices, it may affect the normal operation of the device.

[0039] Therefore, if the method for obtaining the special effects material after responding to a user selection operation is to upload it from the local machine, then first, the size of the media content selected by the user can be detected, and if the size does not exceed a preset size, the special effects material uploaded from the local machine can be obtained, and / or first, the resolution of the media content selected by the user can be detected, and if the resolution does not exceed a preset resolution, the special effects material uploaded from the local machine can be obtained.

[0040] For step 102, the face image may refer to a reference image and / or an actual face image, and can be specifically determined by the type of special effect.

[0041] When displaying a preview of a special effect on the special effects editing page, the preview of the special effect generated based on the face image and the special effect material will differ for different special effect types.

[0042] For example, if the special effect material is a background image, the preview effect of the special effect may refer to using the background image as the background and overlaying the face image onto the background image. If the special effect material is a material other than a background image (key point control material), the preview effect of the special effect may refer to displaying the special effect material directly on the face image according to a predetermined position.

[0043] The aforementioned keypoint control material may refer to a material whose display position is related to the face keypoint, and specifically, when the position of the face keypoint changes, the corresponding display position also changes accordingly, and may be, for example, a face sticker.

[0044] Furthermore, if the special effect material is a key point control material, it is usually displayed on top of the face image, so the face image is located in the lowest graph layer, and the special effect material is located in a graph layer above the graph layer on which the face image is placed. If the special effect material is a background image, the background image is located in the lowest graph layer, and the graph layer on which the face image is placed is located above the graph layer on which the background image is placed.

[0045] If the selected special effect material is a background image, the user has already set a background and does not want to display any content other than their face. Therefore, the contour of the user's face can be extracted separately and overlaid on the background image.

[0046] In one possible embodiment, when the selected special effect material is a background image, when generating a preview effect of the special effect, first a reference image (the reference image can be a face image of a standard model) is obtained, then human body contour information in the reference image is identified, and based on the human body contour information, a human body region image is separated from the reference image, and the human body region image is superimposed on the background image to obtain the preview effect of the special effect.

[0047] In other possible embodiments, video collected by a camera may be directly used to improve the display effect of the face image. Specifically, when the selected special effect material is a background image, when generating a preview effect of the special effect, first, video collected by a camera is acquired / controlled, then contour information within the video frame of the video is identified, and based on the contour information, the human body region image is separated from the video frame, and the human body region image is superimposed on the background image to obtain the preview effect of the special effect.

[0048] Alternatively, in other possible embodiments, preview effects of special effects generated based on the two embodiments described above can be displayed simultaneously. For example, as shown in Figure 2b, a preview effect of a special effect generated based on a reference image can be displayed at a first position, and a preview effect of a special effect generated based on a face video can be displayed at a second position. The user can switch the display position of the two special effect preview effects by triggering the first position.

[0049] Here, the facial video may be an aesthetically pleasing facial video, for example, a facial video that has undergone cosmetic processing and / or deformation processing.

[0050] In practical applications, the system first acquires the unprocessed original video collected by the camera, then performs different processing steps on the original video using different camera processing modules to obtain the face video. For example, a beauty camera processing module can perform beauty processing, and a face deformation camera processing module can perform face deformation (e.g., enlarged eyes, smaller face).

[0051] When selectively displaying preview effects of the aforementioned special effects, different rendering processes can be performed based on different camera processing modules.

[0052] For example, the main perspective camera processing module identifies the human body contour information within the video frame, separates the human body region image from the video frame based on the human body contour information, and displays the background image by rendering the background image into a texture based on the background orthogonal camera processing module.

[0053] When selectively overlaying the face image onto the background image, the background image is rendered based on the background orthogonal camera processing module to obtain a background texture. The rendered background texture and the video are then input to the main perspective camera processing module, which renders the video to obtain a video texture. Based on the background texture, other textures in the video texture, other than the contour texture corresponding to the contour information, are replaced and rendered to obtain a preview effect of the special effect, thereby realizing the overlay of the human body region image and the background image.

[0054] If the special effect material is a key point control material, the preview effect of the special effect may be to overlay the special effect material onto the face image.

[0055] Specifically, by setting a corresponding display position for the special effect material, when the special effect material is superimposed on the face image, it can be superimposed on the face image according to the corresponding display position. The pre-set display positions corresponding to different special effect materials may be different or the same.

[0056] After displaying a preview of the special effect on the special effect editing page, the user can adjust the display information of the special effect material by triggering the special effect material.

[0057] The aforementioned display information may include, for example, the display position, display size, display direction, etc.

[0058] If the special effects material is a background image, the background image may include some of the subjects. The purpose of using an image containing subjects as a background is to composite a group photograph with the subjects. Therefore, if the subjects are obscured when the background image is rendered, the objective of generating a group photograph may not be achieved.

[0059] Accordingly, in one possible embodiment, when displaying a background image within a special effects preview background, if it is detected that the background image contains a target object, the display information for the background image can be determined based on the target position of the target object in the background image and the size information of the background image, and then the background image can be displayed according to the display information.

[0060] Here, the aforementioned object may be a real object such as a building or a flower, or a virtual object such as a virtual character or a virtual building. The aforementioned display information may include, for example, a horizontal extension ratio, a vertical extension ratio, a trimming area range, etc.

[0061] The target position of the object in the background image can be characterized by region coordinates, which may, for example, refer to the coordinates of the vertices of the smallest rectangle that can enclose the object. When selectively determining the background display information based on the target position of the object in the background image and the size information of the background image, the display ratio of the screen interface, etc., can be further combined.

[0062] For step 103, the target area may be a specific physical part of the face that needs to be displayed individually, such as the eyes, nose, or mouth, or it may be a combination of multiple physical parts, such as the five senses.

[0063] The selection operation on the target area may be a selection operation on the face image within the preview effect of the special effect, or a trigger operation on the target button corresponding to the target area.

[0064] The selection operation performed on the target area of ​​the face image within the preview effect of the special effect may be, for example, a direct selection operation on the face image, or for example, a tracing selection operation on the face image. If the selection operation is a direct selection operation on the face image, it is necessary to first enter the area selection mode by triggering the area selection button before performing the selection operation.

[0065] For example, the special effects editing page may display a "part selection" button, and after the user triggers this button, they can directly select a part by tracing over the face image, thereby enabling free selection of the target area.

[0066] Alternatively, in other possible embodiments, a plurality of triggerable part buttons can be pre-configured, and after detecting that any part button is triggered, that part button can be designated as the target button, and the part corresponding to that part button can be designated as the target part.

[0067] When the system is implemented, after any button is triggered, it is possible to detect the positional information of the target area corresponding to the button within the face image, extract the target area image corresponding to the target area from the face image, and update the preview effect of the special effect based on the target area image. If the face image does not contain the target area, the preview effect of the special effect cannot be updated. Therefore, before responding to the triggered target button, it is possible to first determine whether the face image contains the target area.

[0068] For example, if there are multiple body part buttons, it is possible to determine in advance whether the face image contains the body part corresponding to each body part button. If any body part is included, the body part button corresponding to that body part is set to a triggerable state. If a specific body part is not included, the body part button corresponding to that body part is set to a non-triggerable state.

[0069] Selectively, after responding to a selection operation for the target area, keypoint detection is performed on the target area within the face image to determine the positional information of the target area in the face image, and based on the positional information, a target area image corresponding to the target area is separated from the face image. Next, the target area image is input to a foreground orthogonal camera, and the target area image is displayed by rendering the target area image onto a texture with the foreground orthogonal camera.

[0070] Here, when updating the preview effect of the special effect based on the target area image, different updates can be performed depending on the different blocking conditions of the target area, specifically, situations 1 to 3.

[0071] Situation 1: The special effect material is a background image.

[0072] If the special effects material is a background image, the face image is superimposed on the background image, so that no part of the face image is obscured by the special effects material.

[0073] In this case, when updating the preview effect of the special effect based on the target area image, the target area image may be directly displayed in any other area of ​​the face image other than the initial display area of ​​the target area. For example, if the initial display area of ​​the target area in the face image is area A, the target area image may be directly superimposed in any area other than area A when updating the preview effect of the special effect.

[0074] Alternatively, the target area image may be displayed in a region that satisfies a predetermined relative positional relationship with the initial display region. For example, if the initial display region of the target area in the face image is region A, then when updating the preview effect of the special effect, the target area image can be displayed directly in region B, and region B and region A satisfy a predetermined relative positional relationship.

[0075] In this case, when the target area image is displayed using the two methods described above, the face image can still display the target area; that is, the target area image is not an image separated from the face image, but is obtained by copying the region image corresponding to the target area within the face image.

[0076] Alternatively, in other possible embodiments, when updating the preview effect of the special effect based on the target area image, the display of the face image may be directly canceled and the target area image may be displayed.

[0077] Here, the step of canceling the display of the face image and displaying the target area image may, for example, be adjusting the transparency of other areas of the face image other than the target area image to 100%.

[0078] Selectively, in this embodiment, rendering may be performed using multiple camera processing modules. For example, the target area image may be rendered based on the foreground orthogonal camera processing module to obtain a target area texture, and then the target area texture may be superimposed on the screen texture of the currently displayed special effect preview effect, the screen texture may include a background texture rendered by the background orthogonal camera processing module, and a human body contour texture superimposed on the background texture and rendered based on the primary perspective camera.

[0079] If the face image is a video frame within a collected face video, the display position of the target area image can be moved based on the movement of the display position of the target area in the face video.

[0080] Situation 2: The special effect material is a key point control material, and the target area is not blocked by the special effect material.

[0081] In this case, since the purpose of selecting the target area is to display the target area separately, if the target area is not obscured by the special effects material, in one possible embodiment, prompt information is directly displayed to prompt the user that the target area cannot be displayed separately. For example, the prompt information may be "First, block area xx and then cut out."

[0082] Therefore, in this embodiment, before responding to a selection operation on a target area, it is first determined that the target area is blocked by the special effect material.

[0083] In other possible embodiments, if the special effect material is a keypoint control material and the target area is not obstructed by the special effect material, the relative positional relationship between the target area and the special effect sticker can be preset, and when updating the preview effect of the special effect based on the target area image, the target area image can be directly displayed at the corresponding position according to the relative positional relationship.

[0084] Situation 3: The special effect material is a key point control material, and the target area is blocked by the special effect material.

[0085] In this case, since the target area is obscured by the special effect material, the graph layer on which the special effect material is placed is located above the image on which the face image is placed. When updating the preview effect of the special effect based on the target area image, a new graph layer is created and superimposed on the graph layer on which the special effect material is placed, and the target area image is displayed on the newly created graph layer. The display effect is shown exemplarily in Figure 2c.

[0086] If the aforementioned face image is a reference image, the reference image may dynamically display the display effect of the special effect material, and therefore the reference image and the actual face image can be combined simultaneously.

[0087] When displaying a preview of the special effect, if the special effect material is a keypoint control material, the preview of the special effect can be displayed at a first position on the special effect editing page, and simultaneously, a real-time preview of the special effect, including the special effect material and the face image, can be displayed at a second position on the special effect editing page. The display area at the first position is larger than the display area at the second position.

[0088] After responding to a selection operation for the target area, the display positions of the preview effect of the special effect and the real-time preview effect of the special effect can be swapped in order to display the image of the target area more clearly, that is, the preview effect of the special effect can be displayed at the second position and the real-time preview effect of the special effect can be displayed at the first position.

[0089] In response to updating the preview effect of the special effect based on the target area image, it is also possible to update the real-time preview effect of the special effect based on the target area image.

[0090] Since the faces in the reference image are more standard, if the special effect material is a keypoint control material, a preview effect of the special effect including the reference image can be displayed by default at the first position. In this way, the display position of the keypoint control material can be easily adjusted, and the relative positional relationship between the keypoint control material and the reference image can be accurately set. Because the display area at the first position is larger, the real-time preview effect of the special effect can be displayed more comprehensively and dynamically. Therefore, after selecting a target area, displaying the real-time preview effect of the special effect at the first position enables a comprehensive display of the special effect.

[0091] When displaying the preview effect of the special effect, if the special effect material is a background image, the user may prefer to dynamically check the display effect of the background image when previewing the special effect. Therefore, the face image in the preview effect of the special effect may be an image from the face video, that is, the preview effect of the special effect itself is a preview effect of the special effect generated based on an actual face.

[0092] In one possible embodiment, after updating the preview effect of the special effect based on the target area image, the user can also manually edit the target area image. For example, in response to an editing operation on the target area image, the display information of the edited target area can be determined, and the display format of the target area can be adjusted according to the display information.

[0093] Here, the display information may refer to, for example, the display position, display size, display trigger conditions (such as display trigger by blinking), display direction, etc.

[0094] The types of information included in the above-mentioned multiple display pieces may be the same or different, and can be specifically set based on the actual situation.

[0095] In other possible embodiments, after updating the preview effect of the special effect based on the target area image, the system may also perform a censorship operation on at least some of the area images within the target area image in response to a user trigger operation.

[0096] For example, if the target area includes the five senses, the user can perform a concealment operation on the nose and / or eyes, etc., by performing a trigger operation.

[0097] The trigger operation described herein may be, for example, a trigger operation on a hidden button in a particular part of the device, and other trigger methods are not limited in this disclosure.

[0098] Regarding step 104, The decision operation for the preview effect of the updated special effect may, for example, be a trigger operation for the complete button on the special effect editing page. The preview effect of the updated special effect is the preview effect of the currently displayed special effect.

[0099] In one possible embodiment, if the special effect material is a keypoint control material, the display information may further include the relative positional relationship between the target area image and the special effect material, and the relative positional relationship between the target area image and the special effect material can be determined based on the method shown in Figure 3, comprising steps 301 and 302.

[0100] Step 301: Determine the first position information of the target area image set in the preview effect of the updated special effect, and determine the second position information of the key point control material set in the preview effect of the updated special effect.

[0101] The second position information may be the position information of keypoint control material associated with the target area image. If there are multiple added keypoint control materials, the user can manually select the keypoint control material associated with the target area image. For example, the user can display association guidelines between each keypoint control material and the target area image, and by triggering and selecting the association guidelines, the user can create an association between the target area image and the keypoint control material. Keypoint control materials associated with different target area images do not have to be exactly the same, and the same target area image can be associated with at least one keypoint control material.

[0102] Step 302: The relative positional relationship between the target area image and the special effect material is determined based on the first position information and the second position information.

[0103] In other possible embodiments, the relative positional relationship between the target area image and the special effect material may further relate to the type of special effect material.

[0104] For example, if the special effect material is a foreground sticker, the display position of the special effect material on the screen interface is constant, the relative positional relationship between the target area image and the special effect material is empty, and if the special effect material is a face sticker, the relative positional relationship is determined based on the method shown in Figure 3.

[0105] After generating a target special effect based on the display information of the target area and the special effect material, when the target special effect is applied, it can extract the target area from the facial video collected by the camera, display it according to the display information of the target area, and display the special effect material.

[0106] In the special effect generation method according to the disclosed embodiment, the user can freely select special effect materials, and can preview the special effect by displaying a preview effect of the special effect including the face image and special effect materials on the special effect editing page. Furthermore, the user can freely select a target area within the displayed face image to display only the target area. This enhances the special effect generation method and improves the diversity and quality of the generated special effects.

[0107] As those skilled in the art will understand, in the above-described method for carrying out the invention, the order in which each step is described does not limit the process to a strict execution order, and the specific execution order of each step should be determined by the function and possible internal logic.

[0108] Based on a similar inventive concept, the present disclosure further provides a special effect generating apparatus corresponding to a method for generating special effects. Since the principle of solving the problems of the apparatus in the present disclosure is similar to the method for generating special effects in the present disclosure, the implementation of the apparatus can be seen by referring to the implementation of the method, and further details will not be described.

[0109] Figure 4 is a schematic diagram of the structure of a special effect generating device according to an embodiment of the present disclosure, the device comprising an acquisition module 401, a first display module 402, a second display module 403, and a generation module 404. The acquisition module 401 is used to acquire the selected special effect material in response to a selection operation by the user. The first display module 402 is used on the special effects editing page to display a preview effect of a special effect, including a face image and the special effect material. The second display module 403 is used to respond to a selection operation on a target area, to determine a target area image corresponding to the target area from the face image, and to update the preview effect of the special effect based on the target area image. The generation module 404 is used to generate a target special effect in response to a decision operation on the preview effect of the updated special effect, based on the display information of the target portion within the preview effect of the updated special effect and the special effect material.

[0110] In one possible embodiment, if the selected special effect material is a background image, the first display module 402 further: How to collect video with a camera, A method for obtaining a preview effect of a special effect is used, which involves identifying contour information within the video frames of the video, separating a human body region image from the video frames based on the contour information, and superimposing the human body region image onto the background image to obtain a preview effect of the special effect.

[0111] In one possible embodiment, if the selected special effect material is a background image, the first display module 402 further: A method for determining the display information of the background image based on the target position of the target object in the background image and the size information of the background image, when it is detected that the background image contains a target object, This method is used to display the background image in the preview effect of the special effect based on the method of displaying the background image according to the display information.

[0112] In one possible embodiment, if the special effect material is a keypoint control material, the face image becomes a reference image, and the preview effect of the special effect is displayed at a first position. The second display module 403 further, The second position on the special effects editing page is used to display a real-time preview effect of the special effects, including the special effects material and the face image. After responding to a selection operation on the target area, the second display module 403 further: It is used to display a preview effect of the special effect at the second position and a real-time preview effect of the special effect at the first position, wherein the display area at the first position is larger than the display area at the second position.

[0113] In one possible embodiment, the second display module 403 responds to a selection operation on a target area, Responding to a selection operation performed on a target area of ​​a face image within the preview effect of the aforementioned special effect, or It is used to respond to a trigger operation on a target button corresponding to the aforementioned target area.

[0114] In one possible embodiment, the display information of the target area includes the relative positional relationship between the target area image and the special effect material. The generation module 404 is further used to determine the relative positional relationship between the target area image and the special effect material based on the following method: The first position information of the target area image set in the preview effect of the updated special effect is determined, and the second position information of the special effect material set in the preview effect of the updated special effect is determined. Based on the first and second position information, the relative positional relationship between the target area image and the special effect material is determined.

[0115] In one possible embodiment, after updating the preview effect of the special effect based on the target area image, the second display module further: This is used to respond to an editing operation on the target area image, determine the display information of the target area after editing, and adjust the display format of the target area according to the display information.

[0116] In one possible embodiment, the first display module 402 is A method for rendering the background image based on a background orthogonal camera processing module and obtaining a background texture, A method for inputting the aforementioned background texture and the aforementioned video to the main perspective camera processing module, The main perspective camera processing module is used to generate a preview effect of a special effect based on a method of rendering and obtaining a preview effect of the special effect by replacing other textures other than the contour texture corresponding to the contour information based on the background texture.

[0117] In one possible embodiment, when the second display module 403 updates the preview effect of the special effect based on the target area image, Based on the foreground orthogonal camera processing module, the target area image is rendered to obtain the target area texture. This is used to overlay the target area texture onto the currently displayed screen texture of the preview effect of the special effect.

[0118] The processing processes of each module in the apparatus, and the alternating processes between each module, can be described by referring to the relevant explanations in the above-mentioned embodiment of the method, and will not be described in detail here.

[0119] Based on a similar technological concept, the present disclosure further provides computer equipment. Figure 5 is a schematic diagram of the structure of computer equipment 500 according to the present disclosure. The computer equipment of the present disclosure includes, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The computer equipment shown in Figure 5 is merely an example and does not limit the functions and scope of use of the present disclosure.

[0120] As shown in Figure 5, the computer device 500 may include a processing unit (e.g., a central processor, graphics processor, etc.) 501, which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage device 505 into random access memory (RAM) 503. The RAM 503 further stores various programs and data necessary for the operation of the computer device 500. The processing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0121] Typically, input devices 506, including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, and gyroscopes; output devices 507, including, for example, liquid crystal displays (LCDs), speakers, and oscillators; storage devices 508, including, for example, tapes and hard disks; and communication devices 509 can be connected to the I / O interface 505. The communication devices 509 can allow the computer equipment 500 to exchange data with other devices via wireless or wired communication. Figure 5 illustrates computer equipment 500 with various devices, but as can be understood, it is not necessary to implement or include all of the illustrated devices. Instead, it is possible to implement or include more or less of the devices.

[0122] In particular, according to embodiments of the present disclosure, the process described with reference to the flowchart can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product which includes a computer program contained on a computer-readable medium, the computer program including program code for performing a word recommendation method. In such embodiments, the computer program is downloaded and installed from a network by a communication device 509, or installed from a storage device 505, or installed from a ROM 502. When executed by a processing unit 501, the computer program performs the functions limited to the methods of embodiments of the present disclosure.

[0123] The embodiments of this disclosure further provide a computer-readable storage medium which stores a computer program that, when executed, causes a processor to perform the steps of the method for generating special effects described in the above embodiments of the method. The storage medium may be volatile or non-volatile computer-readable storage medium.

[0124] The present embodiment further provides a computer program product which includes program code, and the instructions included in the program code can be used to perform the steps of the method for generating special effects described in the above embodiment, specifically referencing the above embodiment, which will not be described in detail here.

[0125] The above-described computer program product may be implemented specifically by hardware, software, or a combination thereof. In one selectable embodiment, the computer program product is specifically used as a computer storage medium, and in another selectable embodiment, the computer program product is specifically used as a software product, such as a Software Development Kit (SDK).

[0126] For the convenience and simplification of the explanation, and so that those skilled in the art can clearly understand, the specific operating processes of the systems and apparatus described above can be referenced to the corresponding processes in the above-described method embodiments and are not described in detail here. As can be understood in the various embodiments relating to this disclosure, the disclosed systems, apparatus and methods may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the division of the units is merely a logic function division, and there may be other division methods in actual implementation, and as another example, the units or assemblies may be combined or integrated into other systems, or some features may be ignored or not implemented. On the other hand, the coupling or direct coupling or communication connection between the indicated or considered may be an indirect coupling or communication connection of the apparatus or units by some communication interface, and may be electrical, mechanical or other in form.

[0127] The units described as separating members may or may not be physically separated, and the members shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. In order to achieve the objectives of the solution of this embodiment, some or all of the units can be selected according to the actual needs.

[0128] Furthermore, each functional unit in each embodiment of the present disclosure may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit.

[0129] When the aforementioned functions are implemented in the form of a software function unit and sold or used as an independent product, they can be stored in a single processor-executable non-volatile computer-readable storage medium. Based on this understanding, the essence of the technical solutions of the present disclosure, the portion contributing to the prior art, or a part of the technical solutions is implemented in the form of a software product, which is stored in a single storage medium and contains a plurality of instructions that cause a single computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The storage medium includes any medium capable of storing program code, such as a U disk, a mobile hard disk, read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0130] The above embodiments are merely forms for carrying out the invention of this disclosure and are used to illustrate the technical solutions of this disclosure, not to limit them, and the scope of protection of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the above embodiments, as a person skilled in the art will understand, a person skilled in the art can still make modifications or easily conceivable changes to the technical solutions described in the above embodiments, or make equivalent substitutions to some technical features, within the scope of the art disclosed herein. Such modifications, changes, or substitutions do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the scope of protection of this disclosure. Accordingly, the scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A method for generating special effects, The process involves responding to a user's selection operation and obtaining the selected special effect material, The steps include: displaying a preview effect of the special effect, including the face image and the aforementioned special effect material, on the special effect editing page; The steps include: responding to a selection operation on a target area, determining a target area image corresponding to the target area from the face image, and updating the preview effect of the special effect based on the target area image; A method for generating a special effect, comprising the steps of: generating a target special effect in response to a decision operation on the preview effect of the updated special effect, based on the display information of the target portion within the preview effect of the updated special effect and the special effect material.

2. If the selected special effect material is a background image, the method is as follows: How to collect video with a camera, The method according to claim 1, further comprising: a method for identifying contour information within video frames of the video; separating a human body region image from the video frames based on the contour information; and superimposing the human body region image onto a background image to obtain a preview effect of the special effect; and generating a preview effect of the special effect based on the method for the special effect.

3. If the selected special effect material is a background image, the method is as follows: A method for determining the display information of the background image based on the target position of the target object in the background image and the size information of the background image, when it is detected that the background image contains a target object, The method according to claim 1 or 2, further comprising a method for displaying the background image in accordance with the display information, and displaying the background image in the preview effect of the special effect based on the method.

4. If the special effect material is a keypoint control material, the face image becomes a reference image, and the preview effect of the special effect is displayed at the first position. The aforementioned method, The step further includes displaying a real-time preview effect of the special effect, including the special effect material and the face image, at a second location on the special effect editing page. After the step of responding to a selection operation on a target site, the method The method according to claim 1, further comprising the step of displaying a preview effect of the special effect at the second position and displaying a real-time preview effect of the special effect at the first position, wherein the display area at the first position is larger than the display area at the second position.

5. The step in response to a selection operation on a target site is: A step in response to a selection operation performed on the target area of ​​the face image within the preview effect of the special effect, or The method according to claim 1, further comprising the step of responding to a trigger operation on a target button corresponding to the target portion.

6. The display information of the target area includes the relative positional relationship between the target area image and the special effect material. The method further includes the step of determining the relative positional relationship between the target area image and the special effect material based on the following method: The first position information of the target area image set in the preview effect of the updated special effect is determined, and the second position information of the special effect material set in the preview effect of the updated special effect is determined. The method according to claim 1, wherein the relative positional relationship between the target area image and the special effect material is determined based on the first positional information and the second positional information.

7. After the step of updating the preview effect of the special effect based on the target area image, the method The method according to claim 1, further comprising the steps of determining the display information of the target area after editing in response to an editing operation on the target area image, and adjusting the display format of the target area according to the display information.

8. The step of obtaining a preview effect of the special effect is: The steps include rendering the background image based on the background orthogonal camera processing module to obtain a background texture, The steps include inputting the background texture and the video to the main perspective camera processing module, The method according to claim 2, further comprising the steps of: the main perspective camera processing module replaces other textures other than the contour texture corresponding to the contour information based on the background texture, and rendering to obtain a preview effect of the special effect.

9. The step of updating the preview effect of the special effect based on the target area image is: The steps include rendering the target area image based on the foreground orthogonal camera processing module and obtaining the target area texture, The method according to claim 1, comprising the step of overlaying the target area texture onto the screen texture of the preview effect of the special effect currently displayed.

10. A special effects generating device, An acquisition module configured to respond to user selection operations and acquire selected special effect materials, A first display module is configured to display a preview effect of a special effect, including a face image and the aforementioned special effect material, on the special effects editing page. A second display module is configured to respond to a selection operation on a target area, determine a target area image corresponding to the target area from the face image, and update the preview effect of the special effect based on the target area image. A special effect generation device comprising: a generation module configured to generate a target special effect in response to a decision operation on the preview effect of the updated special effect, based on the display information of the target portion within the preview effect of the updated special effect and the special effect material.

11. A computer device comprising a processor, memory, and a bus, wherein the memory stores machine-readable instructions that can be executed by the processor, and when the computer device is executed, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed, the computer device causes the processor to perform the steps of the method for generating special effects described in any one of claims 1 to 9.

12. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that, when executed, causes a processor to perform the steps of the method for generating special effects described in any one of claims 1 to 9.

Images