Special effect processing method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide a special effect processing method and device, electronic equipment and storage medium. The method comprises: in response to a special effect trigger operation for a target special effect; when there is a filament object rendering of a special effect subject corresponding to the target special effect, determining a special effect guide map of the special effect subject, the special effect guide map comprising a basic model of the special effect subject and a key guide line representing the filament object; rendering the special effect guide map to obtain a target special effect picture of the target special effect and display the target special effect picture, wherein the target special effect picture comprises a filament object formed by rendering the key guide line. By using the method, the rendering of the filament object is simplified to fine rendering based on the key guide line of the filament object, which effectively reduces the rendering calculation amount while ensuring the rendering accuracy of the filament object, thereby ensuring the rendering speed of the filament object; at the same time, the production cost is also effectively reduced, and the difficulty of implementing diversified design of the filament object is reduced.

Description

Technical Field

[0001] This disclosure relates to special effects processing technology, and more particularly to a special effects processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In today's society, more and more users are using image effects to display images, so that image effects can vividly and realistically display image content, improve the expressiveness of images, and make images more lifelike.

[0003] In practical applications, due to the large number and small size of hair, real-time rendering of hair and other filamentous objects is relatively difficult. The existing multi-faceted rendering method requires that multiple faces be close enough together.

[0004] However, existing rendering methods have the following problems: hair rendering requires a huge amount of computation, and due to performance limitations, it is difficult to effectively render long hair effects, which reduces hair rendering performance; at the same time, due to the relatively diverse growth patterns and materials of hair, the production cost is high, making it difficult to achieve arbitrary design effects; in addition, the rendering methods are relatively crude, and it is impossible to achieve exquisite rendering of special effects materials, which reduces the rendering effect. Summary of the Invention

[0005] This disclosure provides a special effects processing method, apparatus, electronic device, and storage medium to achieve fast and refined rendering of filamentary objects in special effects.

[0006] In a first aspect, embodiments of this disclosure provide a special effects processing method, which includes:

[0007] Responding to effect triggering actions targeting specific effects;

[0008] When the main body of the target effect has filamentous objects being rendered, the effect guide map of the main body of the effect is determined. The effect guide map includes the basic model of the main body of the effect and key guide lines representing the filamentous objects.

[0009] The special effects guide image is rendered to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen contains a filamentary object formed after rendering the key guide lines.

[0010] Secondly, this disclosure also provides a special effects processing device, which includes:

[0011] The response module is used to respond to effect triggering operations targeting the effect;

[0012] The guide map determination module is used to determine the special effect guide map of the special effect subject when the special effect subject corresponding to the target special effect has filamentous objects being rendered. The special effect guide map includes the basic model of the special effect subject and key guide lines representing filamentous objects.

[0013] The processing and display module is used to render the special effects guide image to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen includes a filamentary object formed after rendering the key guide lines.

[0014] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0015] One or more processors;

[0016] Storage device for storing one or more programs.

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects processing method as described in any embodiment of this disclosure.

[0018] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, characterized in that the computer-executable instructions, when executed by a computer processor, are used to perform the special effects processing method as described in any embodiment of this disclosure.

[0019] The technical solution of this embodiment first responds to a special effect triggering operation for a target special effect; when the main body of the special effect corresponding to the target special effect has filamentous objects being rendered, a special effect guide map for the main body of the special effect is determined. The special effect guide map includes a basic model of the main body of the special effect and key guide lines representing the filamentous objects; then, the special effect guide map is rendered to obtain and display the target special effect image, wherein the target special effect image contains filamentous objects formed after rendering the key guide lines. The technical solution of this embodiment introduces a special effect guide map, which can first determine a special effect guide map containing basic model information of the main body of the special effect and key information of the filamentous objects when the triggered special effect includes the rendering of filamentous objects, as a coarse rendering of the special effect. Subsequently, the special effect guide map can be directly rendered to obtain a target special effect image that achieves precise rendering of the filamentous objects. The above technical solution differs from existing rendering methods for filamentous objects in special effects. It simplifies the rendering of filamentous objects by performing refined rendering based on the key guide lines of the filamentous objects. While ensuring the rendering accuracy of filamentous objects, it effectively reduces the amount of rendering computation and ensures the rendering speed of filamentous objects. At the same time, the rendering of filamentous objects in this technical solution mainly relies on special effect guide maps containing special effect guide maps that represent the key guide lines of filamentous objects. The process of determining special effect guide maps is simple and easy to implement, which also effectively reduces the production cost of filamentous object design and reduces the difficulty of implementing diversified designs of filamentous objects. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 A schematic flowchart illustrating a special effects processing method provided in an embodiment of this disclosure;

[0022] Figure 2a This is an example diagram illustrating the effect of a special effects processing method that includes coarse rendering information in the main body of the special effects, as provided in an embodiment of this disclosure.

[0023] Figure 2b An example diagram illustrating the rendering effect of the basic model and key guide lines of the special effects subject in a special effects processing method provided in this embodiment of the disclosure;

[0024] Figure 2c An example diagram of the target special effects image of the subject of the special effects in a special effects processing method provided in an embodiment of this disclosure;

[0025] Figure 3 A flowchart illustrating another special effects processing method provided in an embodiment of this disclosure;

[0026] Figure 4 This is a schematic diagram of a special effects processing device provided in an embodiment of the present disclosure;

[0027] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0030] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0032] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0033] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0034] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0035] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0036] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0037] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0038] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0039] Figure 1 This is a schematic diagram of a special effects processing method provided in an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to situations where there are special effects processing and rendering of filamentous objects. The method can be executed by a special effects processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, a personal computer (PC) or a server.

[0040] like Figure 1 As shown, the method in this embodiment may specifically include:

[0041] S110, Responding to effect triggering operations for target effects.

[0042] It is clear that more and more users are using image effects to display images, aiming to vividly and realistically showcase image content, enhance image expressiveness, and make images more lifelike. The effect rendering processing method can be integrated into electronic devices such as mobile terminals and PCs. When an effect trigger operation is received, the electronic device can display the corresponding effect image. Here, the effect trigger operation can be understood as the operation used to activate the effect processing function after triggering it. The target effect can be understood as the effect to be applied. Preferably, the effect image in this embodiment is a three-dimensional image effect. Specifically, the target effect can be an effect to be applied that includes the main body of the effect.

[0043] In this embodiment of the disclosure, prior to the special effects processing operation, the process may further include: receiving a special effects trigger operation for the target special effects. The triggering method for the special effects trigger operation can be varied. Optionally, receiving the special effects trigger operation may include, but is not limited to: receiving a special effects trigger operation acting on a preset special effects trigger control, wherein the special effects trigger control can be a virtual control element set on the application interface, for example, the virtual control element includes at least one of a special effects trigger button, a special effects trigger selection menu, and a special effects trigger slider; or, receiving sound information collected by a sound acquisition device for enabling the special effects; or, receiving action information for enabling the special effects (such as hand action information, head action information, or limb action information); or, receiving a special effects enable command for enabling the special effects, etc.

[0044] Specifically, in response to an effect trigger operation targeting a specific effect, the subject to which the target effect is to be applied is determined. For example, if the goal is to display a wolf effect, the wolf effect can be considered the target effect.

[0045] S120. When the main body of the target special effect has a filamentous object being rendered, the special effect guide map of the main body of the special effect is determined. The special effect guide map includes the basic model of the main body of the special effect and key guide lines representing the filamentous object.

[0046] Considering that in practical applications, some target effects contain hair, and due to the large number and small size of hair, real-time rendering of hair and other filamentous objects is relatively difficult, the technical solution provided in this embodiment is mainly used to realize how to perform exquisite rendering of filamentous objects in order to obtain the special effect image of the target effect.

[0047] In this embodiment, the target effect needs to be determined first. Based on the received effect trigger operation, the desired effect is determined, including information such as the main body of the effect. When the effect trigger operation is performed, it is known what kind of effect will be displayed, that is, there is a prediction of the target effect. Based on this prediction, it is possible to extract what the main body of the effect needs to be known to achieve this prediction. After determining the main body of the effect, it can be determined that the main body of the effect should be displayed in the form of the target effect. Similarly, when the main body of the effect is to be displayed in the form of the target effect, it is necessary to determine what objects are included in the display and whether there are any filamentous objects.

[0048] In this context, the main body of the special effect can be understood as the object on which the target special effect acts. A target special effect must have a main body that achieves or displays the effect; in other words, a target special effect can include, but is not limited to, the main body. A target special effect can also include other special effect elements besides the main body. The question also arises whether the target special effect will contain filamentous objects that compose the main body of the special effect.

[0049] It should be noted that the specific content of the target effect is not limited in this embodiment. As long as the target effect includes a main body, and the main body includes filamentous objects, it is acceptable. For example, the target effect could be a virtual hair effect composed of multiple hair strands, a whisk effect composed of multiple whisk strands, or a tassel effect composed of multiple threads, etc. Preferably, the filamentous object is hair.

[0050] It's important to understand that after responding to an effect trigger operation targeting the target effect, it's also necessary to determine whether there are filamentous objects being rendered on the main body of the effect. When there is a need to render filamentous objects, the effect guide image of the main body of the effect needs to be further determined. For example, if the rendered image needs to present a realistic 3D rendering of a wolf, then the desired final effect can be understood as the target effect. The wolf can be considered the main body of the target effect, and the wolf's fur can be understood as the presence of filamentous objects that need to be rendered on the main body of the effect. Therefore, it can be assumed that the main body of the target effect contains filamentous objects being rendered.

[0051] In this embodiment, the special effects guide map includes a base model of the special effects subject and key guide lines representing filamentous objects. The base model of the special effects subject can be understood as a model of the special effects subject obtained from its basic parameters; for example, the base model may include the outline and structure of the special effects subject. Since the special effects subject involves rendering filamentous objects, key guide lines representing these objects are needed. For example, key guide lines can be understood as guide lines obtained from coarse rendering of key hairs, lines, etc., of the special effects subject. With the base model and key guide lines obtained, a coarse rendering of the base model can be performed based on these parameters to obtain the special effects guide map. Based on the base model, key guide lines, and pre-configured coarse rendering parameters for the special effects subject, a coarse rendering of the special effects subject is performed, resulting in the special effects guide map.

[0052] The process of generating the special effects guide map for the main body of the special effects is equivalent to performing a coarse rendering of the main body of the special effects. This coarse rendering yields the special effects guide map. It's important to understand that when a special effects trigger operation is received, the terminal device can quickly and in real-time perform a coarse rendering of the main body of the special effects, rapidly obtaining the special effects guide map. In this embodiment, when the target special effects require rendering of filamentous objects, the special effects guide map can be input into a pre-trained filamentous object rendering model to output the target special effects image. Before inputting into the model, a special effects guide map is needed as input information. The special effects guide map includes a hair guide map and a segmentation guide map. The hair guide map can be obtained based on pre-set key hair guide line information combined with the basic model. The segmentation guide map can be obtained based on deformation parameters, lighting parameters, simulation parameters combined with the basic model. For example, assuming the main body of the special effects is a wolf, the deformation parameters could be the main body opening its mouth wide, its eyes wide open, etc. The hair guide map and the segmentation guide map together constitute the special effects guide map, serving as the input information for the model.

[0053] In this embodiment, when the terminal device performs special effects rendering, it can obtain the basic 3D model of the main subject of the special effects from the material library. For example, to achieve realistic rendering of a wolf, a 3D model of the wolf is first required. Based on the 3D model of the wolf, simple materials, such as some line information, are combined, and then the line information and the 3D model are rendered to form a fur guide map. Similarly, a segmentation map refers to the different features of different areas such as eyes, outlines, and teeth presented in different colors and shapes after the most basic model is obtained. This map is called a segmentation guide map. This segmentation guide map is obtained by rendering the basic model based on some parameters. After obtaining the 3D model, other parameter information can be used, such as knowing where the teeth are located and what shape the teeth are presented. These deformation parameters are combined with the 3D model to form a segmentation guide map. Suppose we want to create an image of a wolf with its mouth open. The model might be a wolf model with its mouth closed. The deformation parameters of the wolf with its mouth open are given to the 3D model. By adjusting the model through these deformation parameters, the wolf is presented in the form of an open mouth. The resulting image can be called a segmentation guide map.

[0054] Specifically, upon receiving an effect trigger operation for a target effect, it is necessary to determine the main body of the target effect and further determine whether the main body of the effect contains filamentary objects for rendering. If filamentary objects are rendered on the main body of the target effect, a coarse rendering is performed based on parameter information, the basic model, etc., to determine the effect guide image of the main body of the effect.

[0055] S130. Render the special effect guide image to obtain and display the target special effect screen of the target special effect, wherein the target special effect screen includes a filamentary object formed after rendering the key guide lines.

[0056] In this embodiment, this step is equivalent to performing precise rendering of the main body of the special effects based on the special effects guide map. The original special effects guide map already contains the basic model and key guide line information. Based on this, the filamentous objects are further enriched, specifically by rendering more, finer, and more detailed yarns on the main body of the special effects. The special effects guide map can be understood as an image obtained by coarsely rendering the basic model based on the key guide line information. The target special effects image can be understood as the filamentous objects formed by further precise rendering of the key guide lines of the main body of the special effects based on the special effects guide map. For example, the special effects guide map can show the display color, display shape, and display position of the main body of the special effects, as well as the key guide lines representing the filamentous objects. Further rendering of the special effects guide map can be understood as further rendering of the key guide lines, making the key guide lines richer and finer.

[0057] For example, one way to render the special effects guide image to obtain the target special effects screen is as follows: based on a pre-trained filamentous object rendering model, the special effects guide image is input into the filamentous object rendering model, and the target special effects screen is output. When there is a need to render filamentous objects in the target special effects, in order to render the special effects screen in real time and quickly, with short rendering time and good rendering effect, the rendering method that originally achieved good offline rendering effect is represented in the form of a model. By continuously training this model, a model that can vividly realize rendering is trained, and this model is directly applied to the terminal.

[0058] For example, when there is a need to render filamentary objects, some information can be directly input into the trained model to obtain the desired special effects. In this embodiment, the trained model can be a neural network model, which needs to be trained in advance. The trained neural network model is denoted as the filamentary object rendering model. In this embodiment, there are no specific restrictions on the specific model structure and training method of the filamentary object rendering model. It is understood that other objects in the special effects image besides filamentary objects can be directly represented in the special effects guide image, while filamentary objects need to be represented more realistically in conjunction with the filamentary object rendering model. After obtaining the target special effects image, the target special effects image can be displayed.

[0059] The technical solution of this embodiment first responds to a special effect triggering operation for a target special effect; when the main body of the special effect corresponding to the target special effect has filamentous objects being rendered, a special effect guide map for the main body of the special effect is determined. The special effect guide map includes a basic model of the main body of the special effect and key guide lines representing the filamentous objects; then, the special effect guide map is rendered to obtain and display the target special effect image, wherein the target special effect image contains filamentous objects formed after rendering the key guide lines. The technical solution of this embodiment introduces a special effect guide map, which can first determine a special effect guide map containing basic model information of the main body of the special effect and key information of the filamentous objects when the triggered special effect includes the rendering of filamentous objects, as a coarse rendering of the special effect. Subsequently, the special effect guide map can be directly rendered to obtain a target special effect image that achieves precise rendering of the filamentous objects. The above technical solution differs from existing rendering methods for filamentous objects in special effects. It simplifies the rendering of filamentous objects by performing refined rendering based on the key guide lines of the filamentous objects. While ensuring the rendering accuracy of filamentous objects, it effectively reduces the amount of rendering computation and ensures the rendering speed of filamentous objects. At the same time, the rendering of filamentous objects in this technical solution mainly relies on special effect guide maps containing special effect guide maps that represent the key guide lines of filamentous objects. The process of determining special effect guide maps is simple and easy to implement, which also effectively reduces the production cost of filamentous object design and reduces the difficulty of implementing diversified designs of filamentous objects.

[0060] For example, in order to more clearly illustrate the special effects processing method provided in the embodiments of this disclosure, the following is a specific implementation of the special effects processing flow. Figure 2a This is an example diagram illustrating the effect of a special effects processing method providing an embodiment of this disclosure, where the main body of the special effects contains coarse rendering information. For example... Figure 2a As shown, it can be seen that Figure 2a The image showcases a rough rendering of the wolf as the main subject of the special effects, primarily focusing on the display of morphological parameters such as the outline of the main subject, the rendering of the eyes when they are open, the outline of the teeth, and the shape of the nose.

[0061] Figure 2b This is an example diagram illustrating the rendering effect of the basic model and key guide lines of the special effects subject in a special effects processing method provided in this embodiment of the disclosure. Figure 2b As shown, the image contains a rendering of the base model of the special effects subject and key guide lines. This rendering can be considered as the result of rendering the base model and key guide lines together. The image mainly contains information about the key guide lines, specifically their location, length, and shape. Combined with... Figure 2a and 2b The information shown can be used for initial rendering to obtain a special effects guide image.

[0062] Figure 2c This is an example diagram of the target special effects image of the special effects subject in a special effects processing method provided in an embodiment of this disclosure. For example... Figure 2c As shown, this is a rendering demonstration of the effect after further rendering and processing based on the special effects guide image. It can be seen that the final special effects image not only includes the shape and hair of the main special effects subject, but also enriches the hair, making it denser and more diverse, resulting in a more realistic effect.

[0063] Figure 3 This is a schematic diagram of another special effects processing method provided in this embodiment of the disclosure. This embodiment further explains the steps for determining the special effects guide image of the special effects subject and the steps for determining the target special effects image, as follows: Figure 3 As shown, the method includes:

[0064] S310, Responds to effect triggering operations for target effects.

[0065] S320. Determine the subject of the special effect corresponding to the target special effect.

[0066] In this embodiment, the target effect needs to be determined first. Based on the received effect trigger operation, the desired effect is determined, including information such as the main body of the effect. When the effect trigger operation is performed, it is known what kind of effect will be displayed, that is, there is a prediction of the target effect. Based on this prediction, it is possible to extract what the main body of the effect needs to be known to achieve this prediction. After determining the main body of the effect, it can be determined that the main body of the effect should be displayed in the form of the target effect.

[0067] S330. If the display object of the special effect subject includes a filamentous object, then it is determined that a filamentous object is rendered on the special effect subject.

[0068] In this step, when the main body of the special effect is to be displayed with the target special effect, it is necessary to determine which objects are included in the display and whether filamentous objects exist. For example, if the main body of the special effect is a wolf, we can determine the current form of the wolf, such as an open mouth and wide eyes. In addition, when it is determined that the main body of the special effect is a wolf, we can clearly know that the material properties configured for the wolf include fur. That is, when displaying the target special effect corresponding to the main body of the special effect, we know whether the displayed special effects include the filamentous object effect. If one of the special effects contains the filamentous object effect, it is determined that there is a filamentous object rendering on the main body of the special effect, that is, there is a special effect rendering requirement on the main body of the special effect.

[0069] S340. Based on the key guide line information and basic model of the special effects subject, generate the special effects guide diagram of the special effects subject, wherein the key guide line information includes the position and / or length of the key guide line.

[0070] In this embodiment, when it is determined that there is filamentary object rendering on the main body of the special effects, that is, when there is a need for filamentary object rendering, it is necessary to generate a special effects guide map for the main body of the special effects based on the key guide line information and the basic model. To obtain the special effects guide map, it is necessary to acquire the key guide line information and the basic model. The basic model refers to the model that needs to be presented in three-dimensional form when different special effects are rendered; a basic three-dimensional model is required here. The key guide line information refers to the filamentary information, i.e., the line information, that needs to be acquired as the basis for filamentary rendering if subsequent filamentary rendering is required. The key guide line information may include the length of the filamentary object and its position on the main body of the special effects. The special effects guide map can be understood as an image obtained by coarsely rendering the basic model based on the guide line information, which can reflect information such as the display color, display shape, and display position of the main body of the special effects.

[0071] Furthermore, based on the basic model and key guide line information of the main body of the special effects, a special effects guide diagram of the main body of the special effects is generated, including:

[0072] a1) Obtain the base model of the special effects subject for 3D modeling, and extract the key guide line information and coarse rendering parameters that are pre-set relative to the special effects subject.

[0073] It's understandable that the special effects guide image is a coarsely rendered image. To generate the special effects guide image for the main special effects, you first need to obtain the basic model and coarse rendering parameters used for 3D modeling of the main special effects. The coarse rendering parameters include key guide line information, as well as coarse rendering parameters related to properties such as material parameters, deformation parameters, and lighting parameters. Key guide line information can include the line's rendering length and its position on the main special effects. Extracting the key guide line information, coarse rendering parameters, and the specific content of the basic model is related to the actual application scenario to be presented. For example, if you want to present special effects on a mobile device or user terminal, these parameters required before the effect is presented can be pre-stored as known information during the design phase, which can also be understood as the special effects material collection or creation phase. Once the target special effects and the main special effects are determined, the data information required for this special effects can be obtained.

[0074] b1) Using the coarse rendering parameters and key guide line information, perform patch rendering on the basic model to obtain the special effects guide image of the main special effects subject.

[0075] Specifically, once the above parameters are obtained, they can be rendered as patches on the base model. Based on the coarse rendering parameters and the base model, a segmentation guide map can be obtained. For example, if there is a deformation in the coarse rendering, the model's position can be adjusted based on the deformation's location information. Based on the key guide line information and the base model, a hair guide map can be obtained. The hair guide map can be understood as an image containing a preliminary coarse rendering of the filamentous object. The segmentation guide map and the hair guide map are combined to form the special effects guide map for the main body of the special effects.

[0076] The above technical solution specifies the steps for generating a special effects guide map for the main body of the special effects based on the basic model and key guide line information. When it is determined that there is a need for rendering filamentous objects, the basic model for 3D modeling of the main body of the special effects is first obtained, and the key guide line information and coarse rendering parameters of the main body of the special effects are extracted. Then, through the coarse rendering parameters and key guide line information, patch rendering is performed on the basic model to obtain the special effects guide map of the main body of the special effects. Patch rendering of the basic model through coarse rendering parameters yields a segmentation guide map, and patch rendering of the basic model through key guide line information yields a hair guide map. The special effects guide map obtained by fusing the segmentation guide map and the hair guide map has a better rendering effect. The guide lines combined with the lighting white model and the semantic segmentation map are used as model input. The key guide line information can be used as supervision information to guide the basic model to generate hair strands. The lighting white model can introduce lighting and contour information, and the semantic segmentation map can improve the generation effect of edges and details. And a more realistic representation of filamentous objects is achieved based on the filamentous object rendering model. It supports hair shaking and lighting transformation based on physical simulation. Compared to other methods of obtaining guide images, the special effects guide images in this technical solution have better rendering quality and more realistic effects. This provides a more accurate input image for the subsequent rendering of target special effects visuals.

[0077] S350. Input the special effects guide image into the pre-trained filamentous object rendering model and output the target special effects image. The filamentous object rendering model is obtained by training through a pre-determined guide image-rendering image sample pair.

[0078] In this embodiment, when there is a need to render filamentary objects in the target special effects, in order to render the special effects quickly and in real time with short rendering time and good rendering effect, the rendering method that originally achieved good offline rendering effect is represented as a model. This model is continuously trained to produce a model that can vividly achieve rendering, and then directly applied to the terminal. When there is a need to render filamentary objects, some information is directly input into this trained model to obtain the desired special effects. In this embodiment, the trained model can be a neural network model, which needs to be trained in advance. The trained neural network model is denoted as the filamentary object rendering model.

[0079] It's important to understand that the filamentary object rendering model is obtained through training on a pre-defined pair of guide image-rendered image samples. This pair includes both guide image samples and rendered image samples. The rendered image samples can be understood as realistic images. By inputting the guide image samples into the neural network model and combining them with the rendered image samples, the filamentary object rendering model can be obtained. In this embodiment, the training method for the filamentary object rendering model is not specifically limited. It is understood that objects other than filamentary objects in the special effects visuals can be directly represented in the special effects guide image; however, filamentary objects require a more realistic representation using the filamentary object rendering model.

[0080] S360, Display the target effect screen of the target effect.

[0081] Specifically, the target effect screen generated by the above steps will be displayed.

[0082] This embodiment specifies the steps for determining the special effects guide map of the special effects subject. When a special effects trigger operation is received for a target special effects, the special effects subject corresponding to the target special effects is first determined. Then, it is determined whether the display object on the special effects subject contains filamentous objects. If it does, it can be determined that there is a rendering requirement for filamentous objects on the special effects subject. Furthermore, based on the key guide line information and basic model of the special effects subject, the special effects guide map of the special effects subject is generated. The technical solution provided by this embodiment can quickly realize the rendering of the special effects guide map, improving the real-time performance of special effects rendering.

[0083] As an optional embodiment of this disclosure, based on the above embodiments, further optimization of the training steps of the filamentary object rendering model includes:

[0084] a2) Construct an initial conditional generative adversarial network model, which includes a generator and a multi-scale discriminator.

[0085] In this embodiment, the concept of adversarial networks is utilized. When training the filamentary object rendering model, a conditional generative adversarial network (GAN) approach is employed. First, an initial GAN ​​model is constructed. This model includes a generator and a discriminator. The generator produces an image based on pre-existing information, and the discriminator determines whether the image is real or fake. The generator's parameters are adjusted based on the discriminator's results, making its output increasingly realistic. Continuously adjusting the discriminator's parameters allows for increasingly accurate identification of fake input images.

[0086] It is understandable that the conditional generative adversarial network (GAN) model in this step is an improved model, and its input information is not multi-dimensional numbers, but images. In this embodiment, an image translation discriminator is used for training. The discriminator's strategy is to use reconstruction to solve low-frequency components, while the GAN is used to solve high-frequency components. On the one hand, traditional loss values ​​are used to make the generated images as similar as possible to the training images, and the GAN is used to construct the details of the high-frequency parts. The idea is that since the GAN is only used to construct high-frequency information, it is not necessary to input the entire image into the discriminator. Instead, the image is first randomly cropped within its range to obtain several image patches of different sizes, and the discriminator then determines whether the image is real or fake.

[0087] This embodiment employs a multi-scale discriminator, which is constructed based on three scales. Each of the three scale discriminators is an independent discriminator, collectively forming the multi-scale discriminator. The multi-scale discriminator can be understood as performing three discriminations. Before the original image is input into the discriminator, it is randomly cropped within the image range to obtain several image blocks of different sizes, maintaining three discriminators with 3, 2, and 1 layers respectively. Each time the image is input into a discriminator with fewer layers, it is downsampled; for example, a downsampling function is used to downsample the image. The 3-layer discriminator directly inputs the original image, the 2-layer discriminator inputs a half-sized image, and the 1-layer discriminator inputs a quarter-sized image. Each discriminator has an output, and the average of the outputs across all scales of an image is used as the output result. The corresponding loss value can also be calculated. For example, the loss value calculation can be similar to the implementation of a switching encoder; no specific limitations are imposed here.

[0088] b2) Obtain a training sample set, which includes at least one set of sample image pairs, each set of sample image pairs including a sample guide image and a sample rendering image.

[0089] In this step, the conditional generative adversarial network (GAN) model can only be trained based on the training sample set. This training sample set is not random; the image pairs included in the training sample set consist of a lead image and a rendered image. The lead image and the rendered image in a sample image pair are rendered for the same subject. The lead image and the rendered image can be understood as different degrees of rendering of the same subject. The lead image is a coarse rendering, while the rendered image is a fine rendering using other engine tools to achieve better results; the rendered image can be understood as a realistic image.

[0090] c2) Based on the sample image pairs, train the generator and the multi-scale discriminator to obtain the filamentous object rendering model.

[0091] In this step, the sample image pair includes a sample guide image and a sample rendering image. The sample guide image is input into the generator to obtain the generator's output. Then, the generator's output, the sample guide image, and the sample rendering image are used together as input to the discriminator. The parameters of the generator and discriminator are adjusted based on the discriminator's output to train the generator and discriminator to meet the accuracy requirements. The trained generator that meets the accuracy requirements is then used as the filamentary object rendering model.

[0092] This optional embodiment introduces a multi-scale discriminator based on the structural similarity index loss and generative adversarial network loss. The discriminator is trained using graphs of the three scales mentioned above, making it more accurate. It improves the detail representation of hair because the input dimensionality is greatly reduced, resulting in fewer parameters and faster computation speed than directly inputting a single graph. Furthermore, it can compute graphs of any size.

[0093] Furthermore, the step of training the generator and multi-scale discriminator based on the sample image pairs to obtain the filamentous object rendering model can be described as follows:

[0094] c21) Use the sample guide map in the sample image pair as the input data of the generator.

[0095] Specifically, the sample guide map in the sample image pair is used as the input data of the generator. The generator can render the sample guide map to obtain the generator's output result.

[0096] c22) The output of the generator and the sample guide map and sample rendering map in the sample image pair constitute the two sets of input data for the multi-scale discriminator.

[0097] Specifically, the generator's output and the sample guide map are used as one set of data, and the sample rendering map and sample guide map in the sample image pair are used as another set of data. The two sets of data together constitute the two sets of input data for the multi-scale discriminator, which are then input into the multi-scale discriminator.

[0098] c23) Based on the output results of the multi-scale discriminator relative to the two sets of input data, and in conjunction with a pre-given loss function, adjust the parameters of the generator and the multi-scale discriminator respectively.

[0099] Specifically, the two sets of input data mentioned above are used as input to the discriminator. After running the program, output results are generated, and these parameters can be adjusted based on the loss function's results. It's important to understand that the pre-defined loss function adjusts both the generator and multi-scale discriminator parameters. The adversarial concept in the Conditional Generative Adversarial Network (GAN) model aims to make the generated rendered image more closely resemble the real image, thus enabling the discriminator to more accurately distinguish between real and fake images. The pre-defined loss function is applied to both the generator and the discriminator; their objectives differ, and therefore their parameter results also differ. In this step, multiple loss functions are used to adjust the parameters.

[0100] c24) After the training iteration ends, the trained generator is used as the rendering model for the filamentary object.

[0101] The iteration termination condition can be understood as follows: the information input into the generator reaches a set precision in the output image to obtain a realistic rendered image; the rendered image and the special effects guide image are input into the discriminator, and the discrimination result reaches a set precision, which can accurately distinguish between real and fake rendered images. If both the generator and the discriminator reach the set precision, the trained generator can be used as a rendering model for filamentary objects for subsequent rendering of filamentary objects.

[0102] This optional embodiment specifies the training steps for a filamentary object rendering model. In scenarios with high-resolution network structures, group normalization is introduced to enhance the stability of the model's performance and reduce flickering in high-frequency scenarios such as hair rendering.

[0103] Furthermore, the generator is represented by a given first network structure, and the multi-scale discriminator is represented by a given second network structure; the loss function includes: a generative adversarial loss function, a learning-aware image patch similarity loss function, and a random image patch loss function.

[0104] Among them, the high-resolution network structure proposed for the 2D human pose estimation task is called the HRNet structure, and the U-shaped network structure is called the UNet structure. The first network structure can be either the HRNet structure or the UNet structure. These two different network structures are used in different scenarios. The two structures have different characteristics: HRNet has a larger computational load but better performance, and is suitable for accelerating rendering and real-time interactive scenarios on computers; the HRNet structure is relatively complex but has better accuracy and realism, and is not very suitable for mobile terminals; UNet can be compressed to a smaller computational load, and is suitable for deploying real-time versions on mobile devices.

[0105] In this embodiment, the model employs an image translation mode, which is essentially a conditional generative adversarial network (GAN) model. The specific principle is as follows: A GAN model is trained to map a contour image to a photograph. The discriminator learns to classify fake images (synthesized by the generator) and real image sets. The generator learns to deceive the discriminator. Unlike ordinary GAN models, where both the generator and discriminator observe the input contour image and either the generated image or the real image, ordinary GAN models directly input either the generated image or the real image.

[0106] This optional embodiment specifies the training steps for the filamentary object rendering model. Based on the structural similarity index loss and generative adversarial network loss, a multi-scale discriminator is introduced. Training the discriminator on graphs of the above three scales makes it more accurate, improving the detail representation of hair. In the HRNet scenario, group normalization is introduced to enhance the stability of the model's performance and reduce flickering in high-frequency scenarios such as hair rendering.

[0107] Optionally, the sample guide map and sample rendering map in the sample image pair are determined by rendering using a pre-given rendering engine tool based on the same rendering parameters.

[0108] In this step, rendering is performed on an offline rendering engine tool to achieve more accurate rendering. Rendering parameters can include camera parameters, lighting parameters, physical parameters, etc. To ensure rendering consistency, the base model needs to be rendered using the same rendering parameters, resulting in sample guide images and sample rendered images. Using the same parameters—that is, ensuring consistency in the rendering of attributes, shape, lighting, etc.—ensures that the sample guide images and sample rendered images are aligned. Sample guide images and sample rendered images with the same parameters are then paired to form sample image pairs. For example, assuming the wolf's special effects rendering shows the wolf with its mouth open, then the wolf in both the sample guide image and the sample rendered image will also have its mouth open, and the open shape will be identical.

[0109] Considering that achieving accurate image rendering is a relatively time-consuming process, not achievable in real-time, this step trades time for rendering quality to obtain a precise sample rendering image. Since the effect guide image can be generated quickly in real-time without consuming too much time, it can be directly input into the pre-trained filamentary object rendering model. This allows for the immediate generation of a target effect image without much delay, enabling fast real-time rendering of filamentary objects and improving effect rendering performance.

[0110] Furthermore, the steps for determining the sample guide image and sample rendering image in the sample image pairing include:

[0111] a3) Obtain the sample body containing filamentous objects and the sample rendering parameters, including camera parameters, lighting parameters and physical simulation deformation parameters.

[0112] Specifically, find the main sample containing filamentous objects and obtain the sample rendering parameters, including camera parameters, lighting parameters, and physical simulation deformation parameters.

[0113] b3) By combining the key guide line information of the sample body with the sample rendering parameters, patch rendering is performed on the sample modeling model of the sample body to obtain the sample guide map of the sample body.

[0114] In this sample image, the sample guide map and the sample rendering map are both rendered from the same sample subject with the same requirements. The two maps have different presentation formats, but the rendered morphological attributes are the same. Specifically, by combining the key guide line information of the sample subject with sample rendering parameters, the sample modeling model of the sample subject is tiled and rendered to obtain a sample hair guide map and a sample segmentation guide map, which together form the sample guide map. The hair guide map mainly reflects the filamentous features of the sample subject. The segmentation guide map presents the shape, deformation, regional location, and lighting parameters of the sample subject.

[0115] c3) On a given offline rendering engine tool, the sample modeling model is rendered offline using the sample rendering parameters to obtain a sample rendering image of the sample body.

[0116] The given offline rendering engine tool can be a rendering engine function. The sample model is rendered offline using sample rendering parameters, and the rendering is performed on the offline rendering engine tool. By configuring some parameters on the offline rendering engine tool, and based on the special effects rendered by the tool, a more realistic and lifelike rendered image can be achieved, serving as the sample's main subject.

[0117] The aforementioned technical solutions, considering that achieving accurate image rendering takes a relatively long time and cannot be achieved in real time, sacrifice time for rendering quality. This step uses traditional graphics rendering algorithms to generate accurate sample rendering images. The sample images generated by this technical solution are used to train a neural network model to obtain a filamentary object rendering model. This model can be directly applied to terminal devices, thereby achieving real-time filamentary object rendering with better rendering effects and improving the performance of rendering effects containing filamentary objects.

[0118] Figure 4 This is a schematic diagram of a special effects processing device provided in an embodiment of the present disclosure, as shown below. Figure 4As shown, the device includes: a response module 410, a guide map determination module 420, and a processing and display module 430.

[0119] The system includes a response module 410, which responds to an effect triggering operation for a target effect; a guide map determination module 420, which determines an effect guide map for the main body of the effect when the main body of the effect corresponding to the target effect has a filamentous object being rendered, wherein the effect guide map includes a basic model of the main body of the effect and key guide lines representing the filamentous object; and a processing and display module 430, which renders the effect guide map to obtain and display the target effect screen of the target effect, wherein the target effect screen includes filamentous objects formed after rendering the key guide lines.

[0120] The technical solution of this embodiment first responds to a special effect triggering operation for a target special effect; when the main body of the special effect corresponding to the target special effect has filamentous objects being rendered, a special effect guide map for the main body of the special effect is determined. The special effect guide map includes a basic model of the main body of the special effect and key guide lines representing the filamentous objects; then, the special effect guide map is rendered to obtain and display the target special effect image, wherein the target special effect image contains filamentous objects formed after rendering the key guide lines. The technical solution of this embodiment introduces a special effect guide map, which can first determine a special effect guide map containing basic model information of the main body of the special effect and key information of the filamentous objects when the triggered special effect includes the rendering of filamentous objects, as a coarse rendering of the special effect. Subsequently, the special effect guide map can be directly rendered to obtain a target special effect image that achieves precise rendering of the filamentous objects. The above technical solution differs from existing rendering methods for filamentous objects in special effects. It simplifies the rendering of filamentous objects by performing refined rendering based on the key guide lines of the filamentous objects. While ensuring the rendering accuracy of filamentous objects, it effectively reduces the amount of rendering computation and ensures the rendering speed of filamentous objects. At the same time, the rendering of filamentous objects in this technical solution mainly relies on special effect guide maps containing special effect guide maps that represent the key guide lines of filamentous objects. The process of determining special effect guide maps is simple and easy to implement, which also effectively reduces the production cost of filamentous object design and reduces the difficulty of implementing diversified designs of filamentous objects.

[0121] Optionally, the guide graph determination module 420 specifically includes:

[0122] The first determining unit is used to determine the subject of the special effect corresponding to the target special effect;

[0123] The second determining unit is used to determine that a filamentous object is rendered on the special effect subject if the display object of the special effect subject includes a filamentous object.

[0124] The guide image generation unit is used to generate a special effects guide image for the special effects subject based on the key guide line information and basic model of the special effects subject.

[0125] Optionally, the guide graph generation unit is specifically used for:

[0126] Obtain the base model of the special effects subject for 3D modeling, and extract the key guide line information and coarse rendering parameters that are pre-set relative to the special effects subject;

[0127] Using the coarse rendering parameters and key guide line information, the basic model is rendered with patches to obtain the special effects guide image of the main special effects subject.

[0128] Optionally, the display module 430 is processed, specifically for:

[0129] The special effects guide image is input into a pre-trained filamentous object rendering model, and the target special effects image is output. The filamentous object rendering model is obtained by training with a pre-determined guide image-rendering image sample pair.

[0130] The target effect screen displays the target effect.

[0131] Optionally, the device further includes a model training module, which specifically includes:

[0132] An initial building unit is used to construct an initial conditional generative adversarial network model, which includes a generator and a multi-scale discriminator.

[0133] A sample acquisition unit is used to acquire a training sample set, wherein the training sample set includes at least one pair of sample images, and each pair of sample images includes a sample guide image and a sample rendering image.

[0134] The training unit is used to train the generator and the multi-scale discriminator based on the sample image pairs to obtain the filamentous object rendering model.

[0135] Optional, training units, which can be used specifically for:

[0136] The sample guide map in the sample image pair is used as the input data of the generator;

[0137] The output of the generator and the sample guide map and sample rendering map in the sample image pair constitute the two sets of input data for the multi-scale discriminator.

[0138] Based on the output of the multi-scale discriminator relative to the two sets of input data, and in conjunction with a pre-given loss function, the parameters of the generator and the multi-scale discriminator are adjusted respectively.

[0139] After the training iteration ends, the trained generator is used as the rendering model for the filamentary object.

[0140] Optionally, the generator is represented by a given first network structure, and the multi-scale discriminator is represented by a given second network structure;

[0141] The loss functions include: generative adversarial loss function, learning-aware image patch similarity loss function, and random image patch loss function.

[0142] Optionally, the sample guide map and sample rendering map in the sample image pair are determined by rendering using a pre-given rendering engine tool based on the same rendering parameters.

[0143] Optionally, the filamentous object is hair.

[0144] The special effects processing apparatus provided in this disclosure can execute the special effects processing methods provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the methods.

[0145] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0146] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0147] like Figure 5As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0148] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0149] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0150] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0151] The electronic device provided in this embodiment and the special effects processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0152] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the special effects processing method provided in the above embodiments.

[0153] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0154] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0155] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0156] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: respond to an effect triggering operation for a target effect;

[0157] When the main body of the target effect has filamentous objects being rendered, the effect guide map of the main body of the effect is determined. The effect guide map includes the basic model of the main body of the effect and key guide lines representing the filamentous objects.

[0158] The special effects guide image is rendered to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen contains a filamentary object formed after rendering the key guide lines.

[0159] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0161] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0162] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0163] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0164] According to one or more embodiments of this disclosure, [Example 1] provides a special effects processing method, the method comprising:

[0165] Responding to effect triggering actions targeting specific effects;

[0166] When the main body of the target effect has filamentous objects being rendered, the effect guide map of the main body of the effect is determined. The effect guide map includes the basic model of the main body of the effect and key guide lines representing the filamentous objects.

[0167] The special effects guide image is rendered to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen contains a filamentary object formed after rendering the key guide lines.

[0168] According to one or more embodiments of this disclosure, [Example 2] provides a special effects processing method, the method comprising:

[0169] Optionally, when there is a filamentary rendering object on the main body of the target effect, determining the effect guide image of the main body of the effect includes:

[0170] Determine the subject of the special effect corresponding to the target special effect;

[0171] If the display object of the main body of the special effect includes filamentous objects, then it is determined that there are filamentous objects being rendered on the main body of the special effect.

[0172] Based on the key guide line information and basic model of the special effects subject, a special effects guide diagram of the special effects subject is generated, wherein the key guide line information includes the position and / or length of the key guide line.

[0173] According to one or more embodiments of this disclosure, [Example 3] provides a special effects processing method, the method comprising:

[0174] Optionally, generating the special effects guide image of the special effects subject based on the basic model and key guide line information of the special effects subject includes:

[0175] Obtain the base model of the special effects subject for 3D modeling, and extract the key guide line information and coarse rendering parameters that are pre-set relative to the special effects subject;

[0176] Using the coarse rendering parameters and key guide line information, the basic model is rendered with patches to obtain the special effects guide image of the main special effects subject.

[0177] According to one or more embodiments of this disclosure, [Example 4] provides a special effects processing method, the method comprising:

[0178] Optionally, the step of rendering the special effects guide image to obtain and display the target special effects screen includes:

[0179] The special effects guide image is input into a pre-trained filamentous object rendering model, and the target special effects image is output. The filamentous object rendering model is obtained by training with a pre-determined guide image-rendering image sample pair.

[0180] The target effect screen displays the target effect.

[0181] According to one or more embodiments of this disclosure, [Example 5] provides a special effects processing method, the method comprising:

[0182] Optionally, the training steps of the filamentary object rendering model include:

[0183] An initial conditional generative adversarial network model is constructed, which includes a generator and a multi-scale discriminator.

[0184] Obtain a training sample set, which includes at least one pair of sample images, and each pair of sample images includes a sample guide image and a sample rendering image;

[0185] The generator and multi-scale discriminator are trained based on the sample image pairs to obtain the filamentous object rendering model.

[0186] According to one or more embodiments of this disclosure, [Example Six] provides a special effects processing method, the method comprising:

[0187] Optionally, training the generator and multi-scale discriminator based on the sample image pairs to obtain the filamentous object rendering model includes:

[0188] The sample guide map in the sample image pair is used as the input data of the generator;

[0189] The output of the generator and the sample guide map and sample rendering map in the sample image pair constitute the two sets of input data for the multi-scale discriminator.

[0190] Based on the output of the multi-scale discriminator relative to the two sets of input data, and in conjunction with a pre-given loss function, the parameters of the generator and the multi-scale discriminator are adjusted respectively.

[0191] After the training iteration ends, the trained generator is used as the rendering model for the filamentary object.

[0192] According to one or more embodiments of this disclosure, [Example Seven] provides a special effects processing method, the method comprising:

[0193] Optionally, the generator is represented by a given first network structure, and the multi-scale discriminator is represented by a given second network structure;

[0194] The loss functions include: generative adversarial loss function, learning-aware image patch similarity loss function, and random image patch loss function.

[0195] According to one or more embodiments of this disclosure, [Example Eight] provides a special effects processing method, the method comprising:

[0196] Optionally, the sample guide map and sample rendering map in the sample image pair are determined by rendering using a pre-given rendering engine tool based on the same rendering parameters.

[0197] According to one or more embodiments of this disclosure, [Example Nine] provides a special effects processing method, the method comprising:

[0198] Optionally, the steps for determining the sample guide map and sample rendering map in the sample image pairing include:

[0199] Obtain the sample body containing filamentous objects and the sample rendering parameters, including camera parameters, lighting parameters, and physical simulation deformation parameters;

[0200] By combining the key guide line information of the sample body with the sample rendering parameters, patch rendering is performed on the sample modeling model of the sample body to obtain the sample guide map of the sample body.

[0201] On a given offline rendering engine tool, the sample modeling model is rendered offline using the sample rendering parameters to obtain a sample rendering image of the sample body.

[0202] According to one or more embodiments of this disclosure, [Example 10] provides a special effects processing method, the method comprising:

[0203] Optionally, the filamentous object is hair.

[0204] According to one or more embodiments of this disclosure, [Example 11] provides a special effects processing apparatus, the apparatus comprising:

[0205] The response module is used to respond to effect triggering operations targeting the effect;

[0206] The guide map determination module is used to determine the special effect guide map of the special effect subject when the special effect subject corresponding to the target special effect has filamentous objects being rendered. The special effect guide map includes the basic model of the special effect subject and key guide lines representing filamentous objects.

[0207] The processing and display module is used to render the special effects guide image to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen includes a filamentary object formed after rendering the key guide lines.

[0208] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0209] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0210] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A special effects processing method, characterized in that, include: Responding to effect triggering actions targeting specific effects; When the main body of the target effect has filamentary objects rendered, an effect guide map for the main body of the effect is determined. The effect guide map includes a filamentary object guide map and a segmentation guide map. The filamentary object guide map is obtained by coarsely rendering the basic model based on key guide line information. The segmentation guide map is obtained by coarsely rendering the basic model based on coarse rendering parameters. The key guide line information includes the position and / or length of the key guide line, and the key guide line information is preset based on the main body of the effect. The coarse rendering parameters include deformation parameters. The special effects guide image is rendered to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen contains a filamentary object formed after rendering the key guide lines.

2. The method according to claim 1, characterized in that, When the main body of the target effect has a filamentary object being rendered, determining the effect guide image of the main body includes: Determine the subject of the special effect corresponding to the target special effect; If the display object of the main body of the special effect includes filamentous objects, then it is determined that there are filamentous objects being rendered on the main body of the special effect. Based on the key guide line information and basic model of the special effects subject, a special effects guide diagram of the special effects subject is generated.

3. The method according to claim 2, characterized in that, The step of generating the special effects guide map for the special effects subject based on the key guide line information and basic model of the special effects subject includes: Obtain the base model of the special effects subject for 3D modeling, and extract the key guide line information and coarse rendering parameters that are pre-set relative to the special effects subject; Using the coarse rendering parameters and key guide line information, the basic model is rendered with patches to obtain the special effects guide image of the main special effects subject.

4. The method according to claim 1, characterized in that, The process of rendering the special effects guide image to obtain and display the target special effects screen includes: The special effects guide image is input into a pre-trained filamentous object rendering model, and the target special effects image is output. The filamentous object rendering model is obtained by training with a pre-determined guide image-rendering image sample pair. The target effect screen displays the target effect.

5. The method according to claim 4, characterized in that, The training steps for the filamentary object rendering model include: An initial conditional generative adversarial network model is constructed, which includes a generator and a multi-scale discriminator. Obtain a training sample set, which includes at least one pair of sample images, and each pair of sample images includes a sample guide image and a sample rendering image; The generator and multi-scale discriminator are trained based on the sample image pairs to obtain the filamentous object rendering model.

6. The method according to claim 5, characterized in that, The step of training the generator and multi-scale discriminator based on the sample image pairs to obtain the filamentous object rendering model includes: The sample guide map in the sample image pair is used as the input data of the generator; The output of the generator and the sample guide map and sample rendering map in the sample image pair constitute the two sets of input data for the multi-scale discriminator. Based on the output of the multi-scale discriminator relative to the two sets of input data, and in conjunction with a pre-given loss function, the parameters of the generator and the multi-scale discriminator are adjusted respectively. After the training iteration ends, the trained generator is used as the rendering model for the filamentary object.

7. The method according to claim 6, characterized in that, The generator is represented by a given first network structure, and the multi-scale discriminator is represented by a given second network structure; The loss functions include: generative adversarial loss function, learning-aware image patch similarity loss function, and random image patch loss function.

8. The method according to claim 5, characterized in that, The sample guide map and sample rendering map in the sample image pair are determined by rendering using a pre-given rendering engine tool based on the same rendering parameters.

9. The method according to any one of claims 1-8, characterized in that, The filamentous object is hair.

10. A special effects processing device, characterized in that, include: The response module is used to respond to effect triggering operations targeting the effect; The guide map determination module is used to determine the effect guide map of the main body of the effect when the main body of the effect corresponding to the target effect has filamentous object rendering. The effect guide map includes a filamentous object guide map and a segmentation guide map. The filamentous object guide map is obtained by coarsely rendering the basic model based on key guide line information. The segmentation guide map is obtained by coarsely rendering the basic model based on coarse rendering parameters. The key guide line information includes the position and / or length of the key guide line, and the key guide line information is preset based on the main body of the effect. The coarse rendering parameters include deformation parameters. The processing and display module is used to render the special effects guide image to obtain and display the target special effects screen of the target special effects, wherein the target special effects screen includes a filamentary object formed after rendering the key guide lines.

11. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects processing method as described in any one of claims 1-9.

12. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the special effects processing method as described in any one of claims 1-9.

Images