Special effect generation method, device and equipment and readable storage medium

Abstract

The invention relates to the technical field of image processing, and discloses a special effect generation method and device, equipment and a readable storage medium. The method comprises the steps that a user interaction interface is displayed, the user interaction interface comprises a display control, and the display control is used for displaying prompt information for triggering a special effect; in response to a special effect triggering operation generated for the prompt information, obtaining characteristic information corresponding to the special effect triggering operation; determining a matched texture resource set based on the characteristic information, and determining a corresponding target texture from the texture resource set; and rendering a corresponding target special effect on the user interaction interface according to the target texture, and mounting the target special effect to a specified part area of the target object. By implementing the technical scheme of the invention, the use experience for the special effect is improved, the generation of the special effect is richer, and the special effect is more diversified.

Description

Technical Field

[0001] The present disclosure relates to the field of image rendering technology, and in particular to a method, device, equipment and readable storage medium for generating special effects. Background Art

[0002] The programmatic generation of textures usually relies on texture synthesis software, such as Substance Designer. These texture synthesis software add features or adjust the input image in the form of a node graph to generate rich materials. In the related art, when using materials generated by programmatic textures as special effects props for application, users are required to manually select to render the special effects props to a specified location, and the display effect is the same for any user, which makes it difficult to match all user types, resulting in inflexible use and display of special effects props, affecting the user experience and richness of special effects props. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a method, device, equipment and readable storage medium for generating special effects to solve the problem of poor user experience and richness of special effect props.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for generating special effects, including: displaying a user interaction interface, the user interaction interface including a display control, the display control being used to display prompt information for triggering the special effect; in response to a special effect trigger operation generated for the prompt information, obtaining characteristic information corresponding to the special effect trigger operation; determining a matching texture resource set based on the characteristic information, and determining a corresponding target texture from the texture resource set; rendering a corresponding target special effect according to the target texture on the user interaction interface, and mounting the target special effect to a specified part area of ​​the target object.

[0005] The method for generating special effects provided by the embodiments of the present disclosure displays a display control for prompt information for triggering special effects on a user interaction interface, so that the target object interacts through the user interaction interface according to the prompt information to obtain corresponding characteristic information, thereby determining a specific target texture from a texture resource set according to the characteristic information, and then performing special effects rendering according to the target texture, and mounting the rendered target special effects on a designated area of ​​the target object. Thus, the display of the target special effects can be triggered by the characteristic information corresponding to the prompt information without manual selection, thereby improving the experience of using special effects. Moreover, for different target objects, the characteristic information obtained according to the prompt information is also different, thereby determining different target textures, rendering different target special effects, and thus displaying different target special effects on the designated area of ​​the target object, thereby enriching the generation of special effects and achieving a diversified effect of special effects.

[0006] In the second aspect, an embodiment of the present disclosure provides a special effect generation device, including: an interface display module, used to display a user interaction interface, the user interaction interface includes a display control, the display control is used to display prompt information for triggering the special effect; a trigger module, used to respond to the special effect trigger operation generated for the prompt information, and obtain characteristic information corresponding to the special effect trigger operation, the characteristic information is used to characterize feedback information generated for the prompt information; a texture determination module, used to determine a matching texture resource set based on the characteristic information, and determine a corresponding target texture from the texture resource set; a special effect mounting module, used to render the corresponding target special effect according to the target texture in the user interaction interface, and mount the target special effect to a specified part area of ​​the target object.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for generating special effects of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method for generating special effects of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 is a flow chart of a method for generating special effects according to an embodiment of the present disclosure;

[0011] Figure 2 is a schematic diagram of a user interaction interface according to an embodiment of the present disclosure;

[0012] Figure 3 is another flow chart of a method for generating special effects according to an embodiment of the present disclosure;

[0013] Figure 4 is another flow chart of a method for generating special effects according to an embodiment of the present disclosure;

[0014] Figure 5 is another schematic diagram of a user interaction interface according to an embodiment of the present disclosure;

[0015] Figure 6 is a flowchart of a method for generating a texture resource set according to an embodiment of the present disclosure;

[0016] Figure 7 is a schematic diagram of generating a texture tile atlas according to an embodiment of the present disclosure;

[0017] Figure 8 is another flow chart of a method for generating a texture resource set according to an embodiment of the present disclosure;

[0018] Fig. 9 is a schematic diagram of encoding of a feature map according to an embodiment of the present disclosure;

[0019] Fig.10 is a schematic diagram of a coding map according to an embodiment of the present disclosure;

[0020] Fig.11 is a schematic diagram of a target texture image according to an embodiment of the present disclosure;

[0021] Fig.12 is a schematic diagram of a geometric texture of cloth according to an embodiment of the present disclosure;

[0022] Fig.13 is a structural block diagram of a device for generating special effects according to an embodiment of the present disclosure;

[0023] Fig.14 It is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0025] In related technologies, when using materials generated by programmatic textures as special effects, users need to manually select to render the special effects to a specified location, and the display effect is the same for any user, which is difficult to match all user types, resulting in inflexible use and display of special effects, affecting the user experience and richness of special effects. Moreover, short video special effects are usually limited by package size, generally less than 10MB. In such a small space, it is difficult to build in rich resources.

[0026] Based on this, this technical solution determines the corresponding characteristic information by displaying prompt information, and then triggers different special effects display by combining different characteristic information, which improves the user experience of special effects and makes the special effects richer. At the same time, the target texture loaded by the electronic device is issued by the cloud server, which can break through the package size limit of the special effect and facilitate the request to obtain rich texture resources from the cloud server during the special effect operation.

[0027] According to an embodiment of the present disclosure, an embodiment of a method for generating special effects is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] In this embodiment, a method for generating special effects is provided, which can be used in the above-mentioned electronic devices, such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for generating special effects according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:

[0029] Step S101, displaying a user interaction interface, wherein the user interaction interface includes a display control, and the display control is used to display prompt information for triggering special effects.

[0030] The user interaction interface is provided by the application installed in the electronic device. When the user triggers the video recording function or the shooting function of the application, the camera of the electronic device will be triggered, and the video recording interface or the shooting interface of the application will be displayed on the screen of the electronic device, and the target object will be displayed on the video recording interface or the shooting interface. The video recording interface or the shooting interface is the user interaction interface.

[0031] Display controls are controls for manual interaction displayed in the user interaction interface. The display controls can be displayed anywhere in the user interaction interface as long as manual interaction can be achieved. Figure 2 As shown, a target object is displayed in the user interaction interface, and the display control can be displayed in the area above the head of the target object.

[0032] The prompt information is text information, picture information, symbol information, etc. displayed in the display control, and the prompt information is a pre-set content used to characterize the characteristics of the target object. Specifically, the prompt information can be saved in a preset file format (such as json) to facilitate mapping the prompt information to the display control.

[0033] Step S102, in response to the special effect triggering operation generated for the prompt information, acquiring characteristic information corresponding to the special effect triggering operation.

[0034] The characteristic information is used to characterize the feedback information generated in response to the prompt information.

[0035] The special effect trigger operation is the operation generated by the target object in response to the prompt information displayed in the display control. Specifically, the special effect trigger operation can be a body movement (such as shaking the head left and right, gestures, etc.), or a voice response. Of course, other triggering methods are also possible, which are not limited here.

[0036] The characteristic information indicates the feedback information generated by the target object in response to the prompt information, and different target objects generate different characteristic information in response to the prompt information. Specifically, different target objects are interested in different contents, so when different target objects view the prompt information, the special effect triggering operations generated in response to the prompt information will also be different.

[0037] Taking body movements as an example, body movements can be captured by a camera of an electronic device, and a user can interact with the prompt information displayed in a display control and the application program through body movements. Accordingly, when the user interaction interface captures the body movements of the target object, it can determine the characteristic information generated for the prompt information according to the body movements of the target object.

[0038] Step S103: determining a matching texture resource set based on the characteristic information, and determining a corresponding target texture from the texture resource set.

[0039] The texture resource set is a large set of texture materials obtained by combining the deep learning model. Specifically, the texture resource set is stored in a cloud server, and each texture material is encoded using a bit mask and has corresponding encoding information. The generation of the texture resource set will be described in detail in the following embodiments.

[0040] The target texture is a texture resource determined from the texture resource set, and the target texture includes special effect material texture and special effect pattern texture, etc. Different characteristic information corresponds to different textures. After obtaining the characteristic information, the texture resource set is accessed based on the characteristic information, and the corresponding target texture is determined from the texture resource set.

[0041] Specifically, the electronic device generates a texture resource request based on the characteristic information, and sends the texture resource request to the cloud server. Accordingly, when the cloud server receives the texture resource request, it accesses the texture resource set, selects the target texture corresponding to the texture resource request from the texture resource set, and sends the target texture to the electronic device. Then, the electronic device loads the target texture sent by the cloud server. In this way, the package size limit of the special effect can be broken through, making it easier to request and obtain rich texture resources from the cloud server during the special effect operation process.

[0042] In a specific embodiment, a scoring algorithm may be used to calculate the score corresponding to the characteristic information. Specifically, the idea of ​​bitmask may be used here to binary encode the characteristic information, and the binary may be converted into a decimal value to obtain the score corresponding to the characteristic information. A texture resource request is sent to the cloud server according to the calculated score. The cloud server may then issue the corresponding target texture from the texture resource set according to the texture resource request.

[0043] Taking the indication information as a question sequence as an example, if there are 4 questions in the question sequence, 16 types of answers can be obtained after bitmask encoding, and the scores of 0 to 15 can be used to send a texture resource request to the cloud server. The cloud server can then determine the corresponding target texture from the texture resource set according to the texture resource request.

[0044] Step S104 , rendering corresponding target special effects according to the target texture in the user interaction interface, and mounting the target special effects to a designated area of ​​the target object.

[0045] The target special effect is a special effect obtained by rendering according to the target texture. The target special effect is rendered in combination with the target texture, and the rendered target special effect is mounted to the specified part area of ​​the target object. For example, a scarf is mounted on the neck area of ​​the target object; a mask is mounted on the face area of ​​the target object; a hat is mounted on the head area of ​​the target object, etc., which are not specifically limited here.

[0046] In some specific implementations, the target object is a real person image of the current user captured by a camera, and the target special effects can be displayed in a specified area of ​​the real person image.

[0047] In some specific implementations, the target object may also be a virtual character image (such as a cartoon image, a sketch image, etc.) simulated based on the user's real image, and the target special effects may be displayed in a specified area of ​​the virtual character image.

[0048] In some specific implementations, the target object may also be a pet, a doll, etc., and the target special effects may be displayed in a designated area of ​​the pet, the doll, etc.

[0049] The method for generating special effects provided in this embodiment displays a display control for prompt information for triggering special effects on a user interaction interface, so that the target object interacts through the user interaction interface according to the prompt information to obtain corresponding characteristic information, thereby determining a specific target texture from a texture resource set according to the characteristic information, and then performing special effects rendering according to the target texture, and mounting the rendered target special effects on a designated area of ​​the target object. Thus, the display of the target special effects can be triggered by the characteristic information corresponding to the prompt information without manual selection, thereby improving the experience of using special effects. Moreover, for different target objects, the characteristic information obtained according to the prompt information is also different, thereby determining different target textures, rendering different target special effects, and thus displaying different target special effects on the designated area of ​​the target object, thereby enriching the generation of special effects and achieving a diversified special effect effect.

[0050] In this embodiment, a method for generating special effects is provided, which can be used in the above-mentioned electronic devices, such as mobile phones, tablet computers, etc. Figure 3 is a flow chart of a method for generating special effects according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:

[0051] Step S201, displaying a user interaction interface, the user interaction interface includes a display control, the display control is used to display prompt information for triggering special effects. For detailed description, please refer to the relevant steps of the above embodiment, which will not be repeated here.

[0052] Step S202, in response to the special effect triggering operation generated for the prompt information, acquiring characteristic information corresponding to the special effect triggering operation, wherein the characteristic information is used to characterize the feedback information generated for the prompt information.

[0053] Specifically, the prompt information includes a question sequence, and each to-be-answered question in the question sequence has corresponding answer information. Accordingly, the above step S202 may include: in response to a selection operation generated for the question sequence, determining a characteristic answer generated for each question in the question sequence based on the selection operation.

[0054] The question sequence is one or more questions displayed in the display control that require the target object to answer. The question sequence is composed of one or more pre-set questions for determining the target texture. Specifically, chatGPT technology can be used here to assist in designing the question sequence.

[0055] The selection operation is an operation for the target object to select an answer to the current question to be answered. Specifically, the application can respond to the target object's selection operation on the answer and determine the characteristic answer corresponding to each question to be answered.

[0056] In some optional implementations, the above steps may specifically include:

[0057] Step S2021: displaying the unanswered questions in the question sequence and multiple alternative answer information corresponding to the unanswered questions in the display control.

[0058] The multiple candidate answer information is composed of different types of words, for example, pairs of perceptual words that are antonyms to each other, such as classic and avant-garde. The question to be answered and its corresponding candidate answer information are displayed in the display control at the same time. Specifically, Figure 2 As shown, the display control is divided into a question area and an answer area, the question area displays the questions to be answered, and the answer area displays the alternative answer information, and different alternative answer information is displayed in different positions in the answer area.

[0059] Step S2022, in response to the selection instruction generated for the question to be answered, select a characteristic answer to the question to be answered from multiple candidate answer information.

[0060] The user triggers the display control by selecting an instruction, and selects the answer information for the question to be answered from the answer area of ​​the display control. The selection instruction can be a body movement instruction or a voice instruction, which is not specifically limited here.

[0061] Taking body movement instructions as an example, the display control can respond to the body movement instructions generated by the target object in response to the question to be answered, and determine the characteristic answer selected by the target object from multiple alternative answer information based on the body movement instructions.

[0062] by Figure 2 For example, when the question displayed in the question area of ​​the display control is "Do you prefer black and white or color?", and the answer area displays the alternative answer information "black and white" and "color", the target object can select "black and white" or "color" by shaking his head. Correspondingly, the target object can respond to the target object's shaking head to determine the answer to the current question to be answered.

[0063] When the question displayed in the question area of ​​the display control is "Do you prefer to maintain the status quo or accept changes", the answer area displays the alternative answer information "Maintain the status quo" and "Accept changes", and the target object can use gestures to select "Maintain the status quo" or "Accept changes". Correspondingly, the display control in the application can respond to the gestures of the target object and determine the answer to the current question to be answered.

[0064] When the question displayed in the question area of ​​the display control is "Do you prefer avant-garde or classic?", and the answer area displays the alternative answer information "avant-garde" and "classic", the target object can answer "avant-garde" or "classic" through voice. Correspondingly, the display control in the application can respond to the voice command of the target object and determine the answer to the current question to be answered.

[0065] In some optional implementations, the above method may further include: when a characteristic answer to the current question to be answered is obtained, sorting the questions according to the question sequence, and displaying the next question to be answered in the display control.

[0066] The display control can display multiple questions to be answered in sequence, each question to be answered is displayed in a pre-set order, and the target object can answer each question to be answered in sequence. The multiple questions to be answered displayed in the display control are randomly selected from the question library.

[0067] Specifically, when a question to be answered is displayed in the display control, the target object can select an answer that suits his or her characteristics from multiple alternative answers through body movements. At the same time, the display control can trigger the display of the next question to be answered, and display the next question to be answered in the question area of ​​the display control.

[0068] Therefore, the method supports displaying each question to be answered in sequence in the display control, and can obtain corresponding characteristic answers, so as to facilitate determining the target texture that meets the current user through the characteristic answers.

[0069] In some optional implementations, the above method may further include:

[0070] Step (1), when a characteristic answer to the current question to be answered is obtained, the next question to be answered associated with the current characteristic answer is determined from the question sequence.

[0071] Step (2), display the next question to be answered in the display control.

[0072] Step (3), when the characteristic answers corresponding to the questions to be answered in the question sequence are obtained, the display controls in the user interaction interface are hidden.

[0073] The number of questions to be answered displayed by the display control is pre-set, such as 3, 4, 5, etc. The number of questions to be answered is not specifically limited here.

[0074] The display control can display multiple questions to be answered in sequence, and the user can answer each question in sequence. In the process of answering, every time the target object selects a characteristic answer, the next question to be answered can be triggered, and the question to be answered is associated with the characteristic answer selected by the target object. For example, the question sequence contains 4 questions to be answered. When the target object selects A1 as the answer to the second question to be answered, the next question to be answered displayed is B1; when the target object selects A2 as the answer to the second question to be answered, the next question to be answered displayed is B2.

[0075] Thus, each question to be answered is associated with the selected answer, so as to determine the target texture suitable for the style of the current target object in combination with the characteristic answer, thereby ensuring that the special effects rendered subsequently can fit the style of the target object.

[0076] When each unanswered question displayed in the display control has obtained the corresponding characteristic answer, the user interaction interface will hide the display control and trigger the loading of the target texture to mount the corresponding target special effect in the specified area of ​​the user's image.

[0077] In some optional implementations, when the user is not interested in the currently displayed target special effect, he or she can re-trigger the display control through gesture operation, body movement or voice operation. Accordingly, the user interaction interface can respond to the user's re-triggering instruction, pop up the display control in the user interaction interface, and display the current question to be answered in the display control, and the current question to be answered is not exactly the same as the previous question to be answered. Thus, the display control can re-acquire the characteristic answer, so that the electronic device reloads the corresponding target texture according to the characteristic answer to generate a new target special effect.

[0078] In the above implementation, after the user has answered all the pending questions, the display control is hidden and the target texture is loaded at the same time, so that the corresponding target texture is selected from the texture resource collection for rendering, thereby improving the rendering immediacy and rendering authenticity of the target special effects.

[0079] Step S203: determine a matching texture resource set based on the characteristic information, and determine a corresponding target texture from the texture resource set. For detailed description, please refer to the relevant steps of the above embodiment, which will not be repeated here.

[0080] Step S204: Render the corresponding target special effect according to the target texture on the user interaction interface, and mount the target special effect to the designated area of ​​the target object. For detailed description, please refer to the relevant steps of the above embodiment, which will not be repeated here.

[0081] The method for generating special effects provided in this embodiment displays multiple alternative answers corresponding to the question to be answered in the display control, so that the user can select the characteristic answer corresponding to the question to be answered from the alternative answers through a selection operation, so as to load the corresponding target texture according to the user's characteristic answer.

[0082] In this embodiment, a method for generating special effects is provided, which can be used in the above-mentioned electronic devices, such as mobile phones, tablet computers, etc. Figure 4 is a flow chart of a method for generating special effects according to an embodiment of the present disclosure, such as Figure 4 As shown, the process includes the following steps:

[0083] Step S301, displaying a user interaction interface, the user interaction interface includes a display control, the display control is used to display prompt information for triggering special effects. For detailed description, please refer to the relevant steps of the above embodiment, which will not be repeated here.

[0084] Step S302, in response to the special effect triggering operation generated for the prompt information, obtain characteristic information corresponding to the special effect triggering operation, wherein the characteristic information is used to characterize the feedback information generated for the prompt information. For detailed description, please refer to the relevant steps of the above embodiment, which will not be repeated here.

[0085] Step S303: determine a matching texture resource set based on the characteristic information, and determine a corresponding target texture from the texture resource set. For detailed description, please refer to the relevant steps of the above embodiment, which will not be repeated here.

[0086] Step S304: Rendering corresponding target special effects according to the target texture in the user interaction interface, and mounting the target special effects to a designated area of ​​the target object.

[0087] Specifically, the above step S304 may include:

[0088] Step S3041, obtaining a target area where the target object is located in the user interaction interface.

[0089] The target area is the display area of ​​the target object in the user interaction interface. Specifically, the application program can call the camera function of the electronic device to determine the target area where the target object is located in the user interaction interface.

[0090] Step S3042, rendering the target special effect in the target area according to the preset rendering method and the target texture, and dynamically mounting the rendered target special effect to the designated part area of ​​the target object.

[0091] The preset rendering mode is a pre-set rendering mode to render the target texture as the target special effect. Specifically, during the rendering process, the light estimation can be performed in combination with the lighting environment of the target object displayed on the user interaction interface to present a virtual rendering effect close to reality for the target special effect. The target texture is rendered according to the light information obtained by the light estimation to obtain the target special effect that meets the real lighting environment.

[0092] Identify the designated area of ​​the target object displayed in the user interaction interface, and dynamically mount the target special effect on the designated area. For example, if the target special effect is a scarf, after the scarf is obtained by rendering according to the target texture, the neck area of ​​the target object is identified through the target detection algorithm, and the scarf is worn on the neck of the target object in a dynamic wearing manner.

[0093] In some optional implementations, the above method may further include: in response to a switching operation on the target special effect, switching the style of the target special effect based on the switching operation.

[0094] There are multiple styles of target special effects generated for target textures. When a user is not satisfied with the current style of the target special effect, the user interaction interface can issue a switching operation to switch the target special effect. Correspondingly, the user interaction interface can respond to the switching operation issued by the user to switch the style of the target special effect.

[0095] Therefore, the method supports the style switching of the target special effect, making it easier for the user to select the special effect style that satisfies him / her, thereby improving the display effect of the target special effect.

[0096] Specifically, the above-mentioned method of switching the target special effect based on the switching operation may include: in response to a limb switching action for the target special effect prop, switching the target special effect sequentially and / or in reverse order according to the limb switching action.

[0097] The body switching action is a body movement made by the user, such as shaking the head to the left, shaking the head to the right, waving to the left, waving to the right, etc., which is not specifically limited here.

[0098] In some specific implementations, the target special effects of various styles rendered for the target texture can be displayed in a preset order. If shaking the head left is for sequential switching, and shaking the head right is for reverse switching, when the user is not satisfied with the target special effect of the current style, he can shake his head left to switch the target special effect, or shake his head right to switch the target special effect.

[0099] Of course, the switching operation is not limited to physical manipulation, but can also be voice switching. In some specific implementations, the target special effects of various styles rendered for the target texture can be displayed in a preset order. If the voice words such as "next" or "switch" are issued, the target special effects can be switched in sequence in a preset order.

[0100] Therefore, different styles of special effects can be obtained by rendering the target texture, and then supporting flexible switching of special effects according to the limb switching movements issued by the target object, without the need to manually operate the user interaction interface, thereby improving the variability of special effects and enriching the target object's experience of different special effects.

[0101] In some optional implementations, the user interaction interface includes an audio control. Accordingly, the method may further include: in response to a trigger operation on the audio control, playing the target audio while mounting the target special effect in a designated area of ​​the target object.

[0102] Audio controls are used to trigger audio playback, such as Figure 5 As shown, an audio control is displayed in the user interaction interface. The user can trigger the audio control to obtain multiple audio information to be selected, and then preview each audio information to be selected to determine the target audio.

[0103] Accordingly, the audio control can respond to a trigger operation issued by the user, enter the audio display interface, display multiple audio information to be selected in the audio display interface, and respond to the user's selection operation on the audio information to be selected to determine the target audio selected by the user.

[0104] Therefore, the corresponding target audio can be played while the special effects are generated, so as to match the special effects with the target audio, making the special effects more vivid and improving the generation quality of the special effects.

[0105] In some optional implementations, the target texture includes a pattern texture. Accordingly, the above method may further include:

[0106] Step (1), in response to a replacement operation on a pattern texture, determining a target pattern texture of a target special effect based on the replacement operation.

[0107] Step (2), re-rendering according to the target pattern texture, generating a target special effect with the target pattern in the designated area.

[0108] If the user is still not satisfied with the pattern texture of the target special effect after multiple switching, he can select the pattern he is interested in from the local gallery of the electronic device as the target pattern texture. The target pattern texture is merged with the material texture corresponding to the target texture to obtain a new texture. The special effect is re-rendered according to the new texture to obtain a target special effect with the target pattern, and the target special effect is mounted to the specified area of ​​the target object.

[0109] In some specific embodiments, when special effects need to be generated in other designated areas of the target object, the user can trigger the generation of special effects for other designated areas of the target object through the user interaction interface to generate special effects with the same target texture in other designated areas.

[0110] For example, if a houndstooth scarf is currently hung on the neck of the target object, the special effects generation for the face area can be triggered to dynamically generate a houndstooth mask. Of course, houndstooth hats, earmuffs, glasses, etc. can also be generated.

[0111] In some specific implementations, when the user needs to generate special effects in other designated areas of the user image, the user interaction interface can be used to trigger the generation of special effects for other designated areas of the target object. At this time, the user interaction interface pops up the display control again, and the user can answer the question on the display control through body movements. Accordingly, after the display control obtains the characteristic answer to the question to be answered, it obtains the target texture corresponding to the characteristic answer and mounts the corresponding special effects in other designated areas.

[0112] For example, if a striped scarf is currently mounted on the neck of the target object, when the special effects for the head area are triggered, the user interaction interface pops up the display control again, and the target object can reply to the question to be answered displayed on the display control through body movements. After the display control obtains the characteristic answer to the question to be answered, it can obtain the target texture corresponding to the characteristic answer again. Assuming that the newly obtained target texture is a houndstooth texture, a hat with a houndstooth texture will be dynamically mounted in the head area.

[0113] Therefore, when the current target special effect does not meet the user's needs, the method supports the user to reselect the target pattern texture to re-render the special effect according to the target pattern texture, further enriching the pattern material of the special effect and improving the user experience of the special effect.

[0114] The special effects generation method provided in this embodiment supports dynamically mounting the target special effects to a specified area of ​​the target object to display the target special effects in the specified area, thereby realizing the dynamic wearing of the target special effects and increasing the authenticity and fun of the special effects mounting.

[0115] In this embodiment, a method for generating a texture resource set is provided, which can be used in cloud servers and other devices. The texture resource set used in the above embodiment is generated by the following method: Figure 6 As shown, the generation process of the texture resource set includes the following steps:

[0116] Step S401: Obtain description information for texture information, where the description information is associated with characteristic information.

[0117] The description information is used to represent the texture features. The description information is input into the cloud server according to actual needs, and the description information is associated with the characteristic information, so that the required target texture can be determined in combination with the association relationship. Specifically, the technician can input a string of description information for the required texture information into the cloud server. Correspondingly, the cloud server can obtain the description information for the texture information input by the technician.

[0118] Step S402: Generate a texture image set corresponding to the description information based on the deep learning model.

[0119] A deep learning model is deployed in the cloud server, such as a machine learning model based on artificial intelligence to generate content. The cloud server can obtain the description information for the texture information and input it into the deep learning model, and output a large number of texture image materials through the deep learning model. The large number of texture image materials constitute a texture image set.

[0120] Step S403: extracting texture features from each texture image in the texture image set to obtain a texture feature atlas corresponding to each texture image.

[0121] In a specific implementation, each texture image in the texture image set is preprocessed by a texture synthesis algorithm to process a non-four-dimensionally continuous texture image into a four-dimensionally continuous texture image, that is, the top and bottom edges of the same texture image are seamlessly connected, and the left and right edges are seamlessly connected.

[0122] With 4 basic blocks ( Figure 7 Taking 1, 2, 3, and 4) in the figure as an example, first, the minimum error path of the overlapping parts of adjacent blocks is calculated in sequence; second, each block is cut along the diagonal to form four new edges, and the area wrapped by the four edges is a basic texture tile; third, the required texture tile atlas is composed in the same way, and then the texture generation rules can be used to programmatically generate textures.

[0123] A preprocessed texture image with a resolution of M*M is selected for texture sampling to extract texture features in the texture image. According to the extracted texture features, 16 texture feature maps with a resolution of N*N can be obtained. The 16 texture feature maps constitute a texture feature map set.

[0124] Since it has been found through testing that texture images with more high-frequency signals have better texture effects, while texture images containing too much illumination information have poorer effects, when performing texture sampling, texture images containing less illumination information are preferred. The resolution can be selected according to actual needs and is not specifically limited here.

[0125] Step S404: generating a target texture image corresponding to each texture feature atlas based on the texture feature maps in each texture feature atlas.

[0126] Step S405: various target texture images are combined into a texture resource set.

[0127] The texture feature maps contained in each texture feature map set are spliced ​​to generate a target texture image corresponding to each texture feature map set. The target texture images corresponding to each texture feature map set are combined to construct a corresponding texture resource set.

[0128] Taking the top edge of the texture feature map as the starting point, assign corresponding attribute characteristic values ​​to each edge of each texture feature map in the texture feature map set, for example, N, E, S, and W represent the top, right, bottom, and left edges, respectively. Two edges represented by the same attribute characteristic value can be seamlessly connected, and thus the texture feature maps in the texture feature map set can be spliced ​​according to the attribute characteristic value to generate a four-way continuous target texture image.

[0129] Different texture feature atlases correspond to different target texture images. Through image processing software, each target texture image is fused and calculated with various types of cloth material textures to generate the texture maps required for PBR materials. At the same time, 3D design software is combined to generate model shapes for special effects. The model shape is bound to the human 3D skeleton, and dynamics are mounted on the model shape. The texture is prefabricated with the texture feature map, and various types of target texture images are obtained to form a texture resource set. Furthermore, the textures in different types of texture resource sets are encoded to facilitate matching with characteristic information.

[0130] Specifically, Figure 8 As shown, the above step S404 may include:

[0131] Step S4041, encoding the edges of the texture feature maps contained in the texture feature map set to obtain a plurality of encoded maps, wherein the encoding is used to characterize the edge state of the texture feature map.

[0132] The number of elements in the texture feature map set is determined by the attributes of the edge. Each edge is encoded according to the possible attributes to obtain a set of encoded maps. Taking the rectangular texture feature map as an example, if each edge has two possible attributes (color, shape), then there are 16 edge states.

[0133] like Fig. 9 As shown, an index value is assigned to each texture feature map. Assuming that the four sides of the rectangular texture feature map have only two possibilities: 0 (absent) and 1 (present), then the 1 state of the "upper side" can be defined as value 1, the 1 state of the "right side" can be defined as value 2, the 1 state of the "lower side" can be defined as value 4, and the 1 state of the "left side" can be defined as value 8. In this way, all the edge states of the texture feature map can be represented.

[0134] Step S4042, splicing multiple coded maps with the same edge state to obtain a target texture image.

[0135] To splice two coded maps, their edge states need to be the same, and splicing them together can produce a seamless target texture image. Fig. 9 As shown in the figure, two coded maps can only be spliced ​​if their corresponding colors are the same. Among them, the top of the coded map numbered 1 is yellow. If you want to splice another coded map on it, the new coded map only needs to have yellow bottom. The following 4, 5, 6, 7, 12, 13, 14, and 15 meet the condition.

[0136] Furthermore, for the encoded map (such as Fig.10 As shown) is decoded to obtain a seamless target texture image (as shown Fig.11 Then, the seamless target texture image is combined with various types of cloth geometric textures (such as Fig.12 As shown) perform fusion calculations to generate the texture map required for the PBR material.

[0137] By encoding the texture feature maps in the texture feature map set, the encoded maps are re-spliced ​​according to the same edge state to obtain the target texture image, thereby ensuring the splicing consistency of the target texture image, eliminating the splicing traces of the target texture image, and improving the generation quality of the target texture image.

[0138] The method for generating special effects provided in this embodiment generates a texture image set using a deep learning model based on the description information obtained for texture information, so as to generate a large number of texture image materials. The texture image material is subjected to texture feature extraction, and a texture feature atlas obtained by combining the texture features is programmatically generated to obtain a texture resource set. In this way, a large number of high-quality materials can be obtained by combining the deep learning model and programmatic texture generation, which improves the material richness and material generation efficiency of the texture resource set, and is convenient for providing a large number of texture materials for special effects, making the generation of special effects more diversified.

[0139] In this embodiment, a special effect generation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0140] This embodiment provides a device for generating special effects, such as Fig.13 As shown, including:

[0141] The interface display module 501 is used to display a user interaction interface, which includes a display control, and the display control is used to display prompt information for triggering special effects.

[0142] The trigger module 502 is used to obtain characteristic information corresponding to the special effect triggering operation in response to the special effect triggering operation generated for the prompt information, wherein the characteristic information is used to characterize the feedback information generated for the prompt information.

[0143] The texture determination module 503 is used to determine a matching texture resource set based on the characteristic information, and determine a corresponding target texture from the texture resource set.

[0144] The special effect mounting module 504 is used to render the corresponding target special effect according to the target texture in the user interaction interface, and mount the target special effect to a designated area of ​​the target object.

[0145] In some optional embodiments, the prompt information includes a question sequence, and the trigger module 502 may include:

[0146] The selection unit is used for responding to a selection operation generated for the question sequence and determining a characteristic answer generated for each question in the question sequence based on the selection operation.

[0147] In some optional embodiments, the selection unit includes:

[0148] The question and answer display subunit is used to display the unanswered questions in the question sequence and multiple alternative answer information corresponding to the unanswered questions in the display control.

[0149] The answer determination subunit is used to select a characteristic answer to the question to be answered from a plurality of candidate answer information in response to a selection instruction generated for the question to be answered.

[0150] In some optional implementations, the trigger module 502 may further include:

[0151] The sequential display unit is used to sort the questions corresponding to the question sequence when a characteristic answer to the current question to be answered is obtained, and display the next question to be answered in the display control.

[0152] In some optional implementations, the trigger module 502 may further include:

[0153] The association unit is used to determine the next question to be answered associated with the current characteristic answer from the question sequence when a characteristic answer to the current question to be answered is obtained.

[0154] The next question display unit is used to display the next question to be answered in the display control.

[0155] The hiding unit is used to hide the display controls of the user interaction interface when the characteristic answers corresponding to the questions to be answered in the question sequence are obtained.

[0156] In some optional implementations, the special effect mounting module 504 may include:

[0157] The area acquisition unit is used to acquire the target area where the target object is located in the user interaction interface.

[0158] The mounting unit is used to render the target special effects in the target area according to the preset rendering method and target texture, and dynamically mount the rendered target special effects to the specified part area of ​​the target object.

[0159] In some optional embodiments, the above device may further include:

[0160] The switching module is used to respond to a switching operation on a target special effect and switch the style of the target special effect based on the switching operation.

[0161] In some optional implementations, the switching module may be specifically used to: in response to a limb switching action on a target special effect prop, perform sequential switching and / or reverse switching on the target special effect according to the limb switching action.

[0162] In some optional implementations, the user interaction interface includes an audio component. Accordingly, the above apparatus may further include:

[0163] The audio trigger module is used to respond to the trigger operation on the audio control and play the target audio while mounting the target special effect in the designated area of ​​the target object.

[0164] In some optional embodiments, the above device may further include:

[0165] The pattern selection module is used to respond to a replacement operation on the pattern texture and determine a target pattern texture of a target special effect based on the replacement operation.

[0166] The rendering module is used to re-render according to the target pattern texture and generate a target special effect with the target pattern in the designated area.

[0167] In some optional embodiments, the above device may further include:

[0168] Texture generation module, used to generate texture resource collection.

[0169] Specifically, the texture generation module may include:

[0170] The description information acquisition unit is used to acquire description information for the texture information, where the description information is associated with the characteristic information.

[0171] The first generating unit is used to generate a texture image set corresponding to the description information based on a deep learning model.

[0172] The feature extraction unit is used to extract texture features from each texture image in the texture image set to obtain a texture feature atlas corresponding to each texture image.

[0173] The second generating unit is used to generate a target texture image corresponding to each texture feature map set based on the texture feature map in each texture feature map set.

[0174] The set generation unit is used to form a texture resource set from various target texture images.

[0175] In some optional implementations, the second generating unit may include:

[0176] The encoding subunit is used to encode the edges of the texture feature maps contained in the texture feature map set to obtain multiple encoded maps, and the encoding is used to represent the edge state of the texture feature map.

[0177] The texture stitching subunit is used to stitch multiple coded textures with the same edge state to obtain a target texture image.

[0178] The further functional description of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0179] The special effect generating device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0180] The special effects generation device provided in this embodiment can trigger the display of the target special effects through the characteristic information corresponding to the prompt information without manual selection, thereby improving the special effects usage experience. Moreover, for different target objects, the characteristic information obtained according to the prompt information is also different, so the target textures determined are different, the target special effects rendered are different, and the target special effects displayed in the designated area of ​​the target object are also different, making the generation of special effects richer and achieving the effect of diversified special effects.

[0181] The present disclosure also provides an electronic device having the above Fig.13 A device for generating the special effects shown.

[0182] See also Fig.14 , Fig.14 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present disclosure, such as Fig.14 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.14 A processor 10 is taken as an example.

[0183] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0184] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0185] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0186] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 604 may also include a combination of the above types of memory.

[0187] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.14 The example of connecting through bus is taken in the following.

[0188] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0189] The computer device also includes a communication interface for the computer device to communicate with other devices or a communication network.

[0190] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0191] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for generating special effects, characterized in that: include: Display a user interaction interface, wherein the user interaction interface includes a display control, and the display control is used to display prompt information for triggering special effects; In response to a special effect triggering operation generated for the prompt information, acquiring characteristic information corresponding to the special effect triggering operation, the characteristic information being used to characterize feedback information generated for the prompt information; Determining a matching texture resource set based on the characteristic information, and determining a corresponding target texture from the texture resource set; The corresponding target special effect is rendered according to the target texture in the user interaction interface, and the target special effect is mounted to a designated area of ​​the target object.

2. The method according to claim 1, characterized in that The prompt information includes a question sequence, and the step of obtaining characteristic information corresponding to the special effect triggering operation in response to the prompt information includes: In response to a selection operation generated for the question sequence, a characteristic answer generated for each question in the question sequence is determined based on the selection operation.

3. The method according to claim 2, characterized in that The step of determining, in response to a selection operation generated for the question sequence, a characteristic answer generated for at least one question in the question sequence based on the selection operation, comprises: Displaying the unanswered questions in the question sequence and a plurality of candidate answer information corresponding to the unanswered questions in the display control; In response to a selection instruction generated for the question to be answered, a characteristic answer to the question to be answered is selected from a plurality of the candidate answer information.

4. The method according to claim 3, characterized in that Also includes: When a characteristic answer to the current question to be answered is obtained, the next question to be answered is displayed in the display control according to the question sorting corresponding to the question sequence.

5. The method according to claim 3, characterized in that: Also includes: When a characteristic answer to the current question to be answered is obtained, determining the next question to be answered associated with the current characteristic answer from the question sequence; Displaying the next question to be answered in the display control; and / or, When the characteristic answers corresponding to the questions to be answered in the question sequence are obtained, the display controls in the user interaction interface are hidden.

6. The method according to claim 1, characterized in that The step of rendering a corresponding target special effect according to the target texture in the user interaction interface and mounting the target special effect to a designated area of ​​the target object includes: Acquire a target area where the target object is located in the user interaction interface; According to the preset rendering method and the target texture, the target special effect is rendered in the target area, and the rendered target special effect is dynamically mounted to a designated area of ​​the target object.

7. The method according to claim 6, characterized in that Also includes: In response to a switching operation on the target special effect, a style of the target special effect is switched based on the switching operation.

8. The method according to claim 7, characterized in that The step of responding to the switching operation instruction for the target special effect and switching the style of the target special effect based on the switching operation instruction includes: In response to a limb switching action for the target special effect, the target special effect is switched sequentially and / or reversely according to the limb switching action.

9. The method according to claim 6, characterized in that The user interaction interface includes an audio control, and the method further includes: In response to a trigger operation on the audio control, the target audio is played while the target special effect is mounted in a designated area of ​​the target object.

10. The method according to any one of claims 6 to 9, characterized in that: The target texture includes a pattern texture. After mounting the target special effect to a designated area of ​​the target object, the method further includes: In response to a replacement operation on the pattern texture, determining a target pattern texture of the target special effect based on the replacement operation; The target pattern texture is re-rendered to generate the target special effect with the target pattern in the designated area.

11. The method according to claim 1, characterized in that The texture resource set is generated in the following manner: Acquire description information for the texture information, wherein the description information is associated with the characteristic information; Generate a set of texture images corresponding to the description information based on a deep learning model; Extracting texture features from each texture image in the texture image set to obtain a texture feature atlas corresponding to each texture image; Based on the texture feature maps in each of the texture feature map sets, generating a target texture image corresponding to each of the texture feature map sets; The target texture images constitute the texture resource set.

12. The method according to claim 11, characterized in that The generating target texture images corresponding to each of the texture feature atlases based on the feature maps in each of the texture feature atlases comprises: Encoding the edges of the texture feature maps contained in the texture feature map set to obtain a plurality of coded maps, wherein the codes are used to characterize the edge states of the texture feature maps; The target texture image is obtained by splicing a plurality of the coded maps having the same edge state.

13. A device for generating special effects, characterized in that: include: An interface display module, used to display a user interaction interface, wherein the user interaction interface includes a display control, and the display control is used to display prompt information for triggering special effects; A trigger module, configured to, in response to a special effect trigger operation generated for the prompt information, obtain characteristic information corresponding to the special effect trigger operation, wherein the characteristic information is used to characterize feedback information generated for the prompt information; A texture determination module, used to determine a matching texture resource set based on the characteristic information, and determine a corresponding target texture from the texture resource set; The special effects mounting module is used to render the corresponding target special effects according to the target texture on the user interaction interface, and mount the target special effects to a designated area of ​​the target object.

14. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for generating special effects described in any one of claims 1 to 12 by executing the computer instructions.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the special effect generation method according to any one of claims 1 to 12.

Images