Special effect generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a special effect generation method and device, electronic equipment and a storage medium, demand information input by a user is received, and the demand information is used for describing a generation requirement of a target special effect based on a natural language; generating a special effect material matched with the image content of the currently displayed target image according to the demand information; and generating a target special effect in the target image based on the special effect material. By converting the generation requirement for the target special effect input by the user into the special effect material matched with the image content of the target image in real time, and further generating the target special effect based on the special effect material, real-time generation of the personalized special effect is realized, so that the content form of the special effect is expanded, the special effect is matched with the image content, and the user experience is improved. And the application effect in the image is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of Internet technology, and in particular to a special effect generation method, device, electronic device and storage medium. Background Art

[0002] Currently, in the application scenarios of media content creation, users can add special effects to images or videos through special effects props provided by the platform or application, so that the images or videos have better visual effects.

[0003] In the prior art, the special effects props provided in the above-mentioned platforms or applications are usually preset in the platforms or applications, and have problems such as fixed styles and limited types, which affect the application effects of special effects in images or videos. Summary of the invention

[0004] The embodiments of the present disclosure provide a special effect generation method, device, electronic device and storage medium to overcome the problem that special effects have fixed styles and limited types.

[0005] In a first aspect, an embodiment of the present disclosure provides a special effect generation method, comprising:

[0006] Receive demand information input by a user, wherein the demand information is used to describe the generation requirements of a target special effect based on natural language; generate special effect materials that match the image content of a currently displayed target image based on the demand information; and generate the target special effect in the target image based on the special effect materials.

[0007] In a second aspect, an embodiment of the present disclosure provides a special effect generating device, including:

[0008] An interactive module, used to receive demand information input by a user, where the demand information is used to describe the generation requirements of a target special effect based on a natural language;

[0009] A generation module, used to generate special effect materials matching the image content of the currently displayed target image according to the demand information;

[0010] A rendering module is used to generate the target special effect in the target image based on the special effect material.

[0011] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0012] The memory stores computer-executable instructions;

[0013] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the special effects generating method described in the first aspect and various possible designs of the first aspect.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the special effects generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0015] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the special effects generation method described in the first aspect and various possible designs of the first aspect.

[0016] The special effect generation method, device, electronic device and storage medium provided in this embodiment receive the demand information input by the user, wherein the demand information is used to describe the generation requirements of the target special effect based on natural language; generate special effect materials that match the image content of the currently displayed target image according to the demand information; and generate the target special effect in the target image based on the special effect materials. By converting the generation requirements for the target special effect input by the user into special effect materials that match the image content of the target image in real time, and then generating the target special effect based on the special effect materials, the real-time generation of personalized special effects is achieved, thereby expanding the content form of the special effect, matching the special effect with the image content, and improving the application effect in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 An application scenario diagram of the special effect generation method provided in the embodiment of the present disclosure;

[0019] Figure 2 A schematic diagram of the process of the special effect generation method provided in the embodiment of the present disclosure Figure 1 ;

[0020] Figure 3 A schematic diagram of an interaction for a user to input demand information provided by an embodiment of the present disclosure;

[0021] Figure 4 for Figure 2 A flowchart of a specific implementation method of step S102 in the illustrated embodiment;

[0022] Figure 5 A schematic diagram of a process for determining a target special effect material provided by an embodiment of the present disclosure;

[0023] Figure 6 for Figure 2 A flowchart of a specific implementation method of step S103 in the illustrated embodiment;

[0024] Figure 7 A schematic diagram of the process of the special effect generation method provided in the embodiment of the present disclosure Figure 2 ;

[0025] Figure 8 for Figure 7 A flowchart of a specific implementation method of step S202 in the illustrated embodiment;

[0026] Fig. 9 for Figure 8 A flowchart of a specific implementation method of step S2022 in the illustrated embodiment;

[0027] Fig.10 A schematic diagram of a process for generating a second prompt word provided by an embodiment of the present disclosure;

[0028] Fig.11 for Figure 7 A flowchart of a specific implementation method of step S205 in the illustrated embodiment;

[0029] Fig.12 A structural block diagram of a special effect generating device provided in an embodiment of the present disclosure;

[0030] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0031] Fig.14 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] The application scenarios of the embodiments of the present disclosure are explained below:

[0035] Figure 1 This is an application scenario diagram of the special effects generation method provided in the embodiment of the present disclosure. The special effects generation method provided in the embodiment of the present disclosure can be applied to applications with special effects editing functions, and more specifically, can be applied to application scenarios of shooting videos and photos with video special effects. The execution subject of this embodiment can be a terminal device running the above-mentioned application with special effects editing function, or a server that deploys the service end corresponding to the above-mentioned application, or other electronic devices that perform similar functions. Figure 1 As shown in , taking a terminal device as an example, the terminal device is, for example, a smart phone. The terminal device shoots a user through a camera unit and displays a real-time image of the photographed user in a viewfinder interface; then, in response to a user operation, a target special effect is triggered in the viewfinder interface, such as the virtual "glasses" shown in the figure, so that the target special effect is displayed in the real-time image. Then, the terminal device can shoot an image or record a video based on the target special effect, thereby generating a special effect video or special effect picture with the above target special effect.

[0036] Of course, the method provided in this embodiment can also be applied to scenarios where non-real-time images are edited. For example, the terminal device can add video effects to the above-mentioned taken pictures or videos by loading the pictures or videos that have been taken and using the special effects props provided by the application.

[0037] In the prior art, in the process of implementing the above-mentioned process of adding target special effects to real-time images or non-real-time images through user operations, it is usually implemented by using special effects props provided in the platform or application, and the user can select one or more special effects props from the special effects props provided in the above-mentioned platform or application to add special effects. However, on the one hand, the process of users manually selecting special effects props is time-consuming and inefficient, affecting the overall efficiency of special effects editing; on the other hand, because special effects props are pre-generated, the styles and types are limited, resulting in users often being unable to obtain target special effects that match the video content or that the user is interested in, thereby affecting the application effect of the special effects in the image. At the same time, the above two problems restrict each other. When the number and styles of preset special effects are small, the application effect of special effects in the image is affected. When the number of preset special effects is unilaterally increased, it will increase the difficulty of user retrieval and affect efficiency.

[0038] The embodiments of the present disclosure provide a special effect generation method to solve the above problems.

[0039] refer to Figure 2 , Figure 2 A schematic diagram of the process of the special effect generation method provided in the embodiment of the present disclosure Figure 1 The method of this embodiment can be applied in a terminal device, and the special effect generation method includes:

[0040] Step S101: receiving demand information input by a user, where the demand information is used to describe generation requirements of a target special effect based on natural language.

[0041] For example, reference Figure 1 In the schematic diagram of the application scenario shown, before the terminal device triggers the target special effect, it is necessary to determine the target special effect based on the user operation. In one possible implementation, the terminal device displays an input control for inputting information in the first interface, such as an editable text box, or a voice input button, and the user inputs characters and voices to the terminal device through the input control so that the terminal device obtains the demand information. The demand information is used to describe the generation requirements of the target special effect based on natural language. For example, the demand information can be a piece of text, and the corresponding text content is "Help me generate a science fiction-style background special effect."

[0042] Figure 3 This is a schematic diagram of an interaction for a user to input demand information provided by an embodiment of the present disclosure. Figure 3 The above step S101 is introduced. Figure 3 As shown, after the terminal device runs the target application, it calls up a viewfinder interface for shooting videos, in which a control #1 for adding special effects is provided. In response to the user's click operation on the control #1, a first interface is displayed in the viewfinder interface, which includes an editable text box edit_text_1 (shown as edit_text_1 in the figure). After that, the user enters text into the editable text box, such as "help me generate a sci-fi style background special effect". The terminal device performs semantic analysis based on the above input text and generates demand information; or directly uses the above input text as demand information to execute subsequent steps.

[0043] Step S102: Generate special effect materials that match the image content of the currently displayed target image according to the demand information.

[0044] Exemplarily, first, the currently displayed target image refers to the image currently displayed in the interactive interface of the terminal device, that is, the image currently to be inserted with the target special effect. Specifically, the currently displayed target image can be, for example, an image captured in real time by a camera unit displayed in the camera viewfinder, or an image in a pre-generated picture or video currently loaded, for example, the target image is a video frame in a pre-loaded video; more specifically, the target image can be a cover image of a video, or a video frame determined by a jump (seek) operation imposed by the user. Based on different application scenarios, the implementation method of the currently displayed target image may be different, and no specific limitation is imposed here.

[0045] Furthermore, after the terminal device obtains the demand information, it processes the demand information to generate special effect materials, and matches the special effect materials with the image content of the currently displayed target image. The special effect materials are media data used to generate special effects, and may include one or more of pictures, sequence frames, videos, audios, etc.

[0046] The image content of the special effects material matches that of the target image, including multiple implementation methods. Taking the case where the special effects material is a picture as an example, for example, the image content of the special effects material matches that of the target image, which may mean that the scale information of the special effects material is consistent with the scale information of the target image, and the scale information represents the proportional relationship between the size of an object in the image and the real size of the object. Alternatively, it may mean that the overall image tone (cold tone or warm tone) of the special effects material is consistent with the overall tone of the target image; or, it may mean that the image style of the special effects material (such as comic style, realistic style) is consistent with the image style of the target image, and so on.

[0047] Specifically, there are multiple implementation methods for generating special effect materials by processing the terminal device based on the demand information. For example, based on the demand information, a matching network resource image is searched from the network, and the network resource image is processed to generate the special effect material. In one possible implementation method, Figure 4 As shown, the specific implementation of step S102 includes:

[0048] Step S1021: Generate a first prompt word according to the demand information, where the first prompt word is used to characterize the target special effect category to which the target special effect belongs.

[0049] Step S1022: According to the first prompt word, a data index is obtained, where the data index is used to indicate a user-generated content library corresponding to the target special effect category.

[0050] Step S1023: Obtain corresponding special effect material according to the data index and the image content of the target image.

[0051] Exemplarily, first, the terminal device generates a first prompt word corresponding to the demand information, such as a descriptive text, to represent the target special effect category to which the target special effect belongs, wherein the first prompt word can be a text representing the semantics of the demand information generated by the language model, or one or more keywords in the demand information. Specifically, the first prompt word includes, for example: "background category", "headwear category", "beauty category", etc.

[0052] Afterwards, based on the preset mapping rules, the first prompt word is summarized and mapped to obtain the corresponding data index, and the data index is used to indicate the user-generated content library corresponding to the target special effect category. Specifically, the data index can be an identifier that represents a specific user-generated content library, or it can be the library address of the user-generated content library. For example. "Background class" is mapped to the corresponding UCG_01 library, and "headwear material" is mapped to the corresponding UCG_02 library. Among them, the user-generated content library in this embodiment is a resource library for storing user-generated content (User Generated Content, UGC). The user-generated content library can be preset inside the terminal device, or it can be stored in a server outside the terminal device or in a network (such as the Internet), which is not limited here. Afterwards, based on the data index, media data that matches the image content of the target image is obtained from the corresponding user-generated content library, thereby obtaining special effect materials.

[0053] In the steps of this embodiment, by converting the demand information into a first prompt word that represents the target special effect category to which the target special effect belongs, and then obtaining user-generated content based on the first prompt word, the external user-generated content is fully utilized to obtain special effect materials, thereby improving the content richness of the special effect materials.

[0054] Furthermore, in a possible implementation, after step S1022, the following is further included:

[0055] Step S1022A: Based on the first prompt word, a first rendering parameter is obtained, where the first rendering parameter is used to characterize the position and / or distribution of a target special effect generated based on the special effect material in the target image.

[0056] Exemplarily, after obtaining the first prompt word representing the target special effect category to which the target special effect belongs, the video materials corresponding to the special effects of different special effect categories correspond to different rendering positions or different position distributions when the special effects are performed. For example, when the target special effect category is "headwear material", the target special effect generated based on the special effect material is at the position of the upper contour of the portrait's head; when the target special effect category is "glasses material", the target special effect generated based on the special effect material is at the position of the contour of the portrait's eyes. When the target special effect category is, for example, "starry sky background special effect", the target special effect generated based on the special effect material "stars" is evenly distributed around the portrait contour.

[0057] The above-mentioned information characterizing the position and / or distribution of the target special effect generated by the special effect material in the target image is the first rendering parameter. The rendering parameter can be pre-generated, and after obtaining the first prompt word, it is mapped to the corresponding first rendering parameter based on the first prompt word, and then in the subsequent process of generating the target special effect, the special effect material is rendered based on the first rendering parameter, so that the special effect material appears at the exact position in the target image, that is, the target special effect is generated in the target image according to the special effect material and the first rendering parameter.

[0058] In this embodiment, a first rendering parameter representing the position and / or distribution of a target special effect generated based on the special effects material in a target image is generated according to a first prompt word. In the subsequent step of generating the target special effect in the target image, the special effects material can be accurately rendered at the corresponding image position based on the first rendering parameter, thereby improving the visual effect of the target special effect, saving the step of manually setting the target special effect position, and improving the efficiency of configuring special effects.

[0059] Furthermore, in a possible implementation, after generating the special effect material, the process further includes:

[0060] Step S102A: Display at least one special effect material in the first interface of the terminal device.

[0061] Step S102B: In response to the user's selection operation, determine the target special effect material.

[0062] For example, Figure 5 A schematic diagram of a process for determining a target special effect material provided by an embodiment of the present disclosure, referring to Figure 5As shown in , after the terminal device obtains the special effect material, at least one special effect material generated in the above steps is displayed in the first interface, which is shown as special effect material A, special effect material B and special effect material C. Afterwards, the user applies a selection operation through the interactive interface, and the terminal device responds to the selection operation and determines one or more of them as target special effect materials to participate in the subsequent target special effect generation step. For example, as shown in the figure, in response to the user's selection operation, special effect material B is determined as the target special effect material, and in the subsequent steps, it is processed based on the target special effect material (that is, the "special effect material" in the subsequent steps is the "target special effect material" determined in this step) to generate the corresponding target special effect.

[0063] Step S103: Generate a target special effect in a target image based on the special effect material.

[0064] Exemplarily, after obtaining the special effects material, the terminal device further processes the special effects material, for example, including processing the size or resolution of the special effects material, cropping the outline of the special effects material, and / or cropping the target image based on the outline of the special effects material, etc. Thereafter, the special effects material is rendered to the target image, thereby displaying the target special effect in the target image, completing the process of adding special effects to the target image.

[0065] In one possible implementation, Figure 6 As shown, the specific implementation of step S103 includes:

[0066] Step S1031: generating a first special effect material and a second special effect material according to the demand information, wherein the first special effect material is pre-generated user-generated content, and the second special effect material is real-time generated model-generated content;

[0067] Step S1032: Generate special effect material according to the combination of the first special effect material and the second special effect material.

[0068] For example, in a possible implementation, the corresponding first special effect material and the second special effect material are obtained respectively according to the demand information, wherein the first special effect material is a pre-generated user-generated content, and the method for obtaining the first special effect material can refer to the above Figure 4The specific implementation steps of step S102 shown are not repeated here. The second special effects material is based on the content generated by the generation model, that is, based on the special effects material generated by the generative artificial intelligence (Artificial Intelligence Generated Content, AIGC) technology. Specifically, for example, the corresponding prompt words can be generated by the demand information, and then the generation model is guided by the prompt words to generate the second special effects material. Other implementation methods for generating the second special effects material will be further introduced in the subsequent embodiments and will not be repeated here. Afterwards, the first special effects material and the second special effects material are combined, such as splicing, superposition, fusion, etc., to generate the special effects material.

[0069] In the steps of this embodiment, by respectively obtaining user-generated content and model-generated content and combining them, the obtained special effects material can have both the specific typicality of the user-generated content and the flexibility of the model-generated content, so that the target special effects finally generated based on the special effects material can have specific meanings and express specific existing information (taking advantage of the specific typicality of the user-generated content), and can also increase diversity and reduce repetition in style (taking advantage of the flexibility of the model-generated content), thereby improving the quality of the target special effects generated based on the special effects material.

[0070] The special effect generation method, device, electronic device and storage medium provided in this embodiment receive the demand information input by the user, and the demand information is used to describe the generation requirements of the target special effect based on natural language; generate special effect materials that match the image content of the currently displayed target image according to the demand information; and generate the target special effect in the target image based on the special effect materials. By converting the generation requirements for the target special effect input by the user into special effect materials that match the image content of the target image in real time, and then generating the target special effect based on the special effect materials, the real-time generation of personalized special effects is achieved, thereby expanding the content form of the special effect, matching the special effect with the image content, and improving the application effect in the image.

[0071] refer to Figure 7 , Figure 7 A schematic diagram of the process of the special effect generation method provided in the embodiment of the present disclosure Figure 2 In this embodiment Figure 2 On the basis of the embodiment shown, steps S102-S103 are further refined, and the target special effects are generated using artificial intelligence generated content (AIGC) technology. The special effects generation method includes:

[0072] Step S201: receiving demand information input by a user, where the demand information is used to describe generation requirements of a target special effect based on natural language.

[0073] Step S202: Generate a second prompt word according to the demand information, and the second prompt word is used to guide the target model to generate special effect materials that meet the generation requirements.

[0074] For example, the second prompt word is a prompt word or a prompt for inputting into the generation model, which is used to clarify tasks or requirements, limit scope or theme, specify format or structure, determine tone or style, and specify key information or elements, so as to guide the target model to generate special effects materials that meet the generation requirements. The target model is the generation model for generating materials mentioned above (refer to the relevant introduction of the second special effects material). For example, Figure 8 As shown, the specific implementation of step S202 includes:

[0075] Step S2021: Obtain image content features of the currently displayed target image.

[0076] Step S2022: Generate a second prompt word based on the image content feature and the demand information.

[0077] Exemplarily, first, for the currently displayed target image, such as the real-time captured image displayed in the camera viewfinder, it is processed to obtain the image content features corresponding to the target image; wherein the image content features can be an identifier representing the image content type, or a feature array representing the image content, wherein each element in the feature array represents the content (category) under a content dimension of the target image, and the image content features can also be a more complex feature matrix, which will not be described in detail. Of course, in another possible implementation, the image content features can be a pixel matrix obtained after processing the pixel points in the target image, such as after pooling processing (resolution reduction), and the specific implementation method can be set as needed, which will not be described in detail here.

[0078] Furthermore, based on the set of image content features and demand information, prompt words are generated to generate second prompt words. The generated prompt words are combined with the image content of the target image while describing the generation requirements, so as to achieve the purpose of matching the special effect material generated based on the second prompt words with the image content of the target image. In one possible implementation, the image content features can be mapped to specific parameters and inserted into the demand information to obtain the second prompt information; in another possible implementation, the second prompt words can be generated by combining the image content features and the demand information through a template for generating prompt words, which will not be described one by one.

[0079] Furthermore, if Fig. 9 As shown, the specific implementation of step S2022 includes:

[0080] Step S2022A: Generate a first text parameter based on the image content feature, where the first text parameter is used to describe the content semantics of the target image.

[0081] Step S2022B: Based on the demand information and in combination with the prompt word template, a prompt word text is generated, wherein the prompt word text includes input parameters, and the prompt word text represents processing logic for processing the input parameters to generate special effect materials that meet the generation requirements.

[0082] Step S2022C: Use the first text parameter as an input parameter, insert the prompt word text, and generate a second prompt word.

[0083] Furthermore, in a possible implementation, firstly, based on the image content feature, a first text parameter representing the content semantics of the target image is mapped. For example, the image content feature is a pixel matrix obtained after image processing of the target image. The image content feature is identified and summarized, and the first text parameter generated is Context_1, and its corresponding parameter value includes the text "jogging in the park", which is a description of the content semantics of the target image. After that, a prompt word text with input parameters is generated by combining the prompt word template and the demand information, and the first text parameter is used as the input parameter, and the prompt word text is inserted to generate the second prompt word.

[0084] Fig.10 A schematic diagram of a process for generating a second prompt word provided in an embodiment of the present disclosure, referring to Fig.10 As shown, exemplarily, first, feature extraction is performed on the target image P1 to obtain the image content feature feature_1 (shown as feature_1 in the figure), and based on the image content feature feature_1, the first text parameter "running in the park" is obtained. Afterwards, the content of the demand information input by the user includes "generate a science fiction style background for me". Based on the demand information and the prompt word template, a prompt word text Info_1 with input parameters is generated, as shown in the figure, and its content includes, for example: "for the scene [input], generate a [science fiction style] picture". The keyword "science fiction style" in the above content is generated by the prompt word template according to the demand information, and the keyword "input" can be considered as a reserved parameter. The prompt word text Info_1 represents the processing logic for processing the input parameter input to generate special effect materials that meet the generation requirements. Afterwards, the first text parameter is used as the input parameter input, and the prompt word text (shown as input = first text parameter in the figure) is inserted, and the content of the obtained second prompt word Info_2 includes "for the scene [running in the park], generate a [science fiction style] picture". Optionally, the prompt word template may be determined based on demand information.

[0085] Step S203: Generate special effect material based on the second prompt word and the target model.

[0086] Exemplarily, after that, based on the generated second prompt word and the corresponding target model, the target model can be guided to generate corresponding content, i.e., special effect material. In the steps of this embodiment, the second prompt word is generated by combining the image content features, and the target model is guided to output the special effect material based on the second prompt word. Since the image content features of the target image are integrated into the second prompt word, the special effect material generated based on the second prompt word can match the image content of the target image, thereby improving the matching degree between the target special effect and the target image on the basis of generating personalized target special effects, and improving the application effect of the special effect in the image.

[0087] Step S204: obtaining a special effect template corresponding to the target special effect, wherein the special effect template is used to process at least one special effect material.

[0088] Step S205: performing image rendering based on the special effect template and the special effect material to generate a target special effect in the target image.

[0089] Exemplarily, after obtaining the special effect material, the special effect template is used to further process the special effect material, thereby obtaining special effect rendering data that matches the target image, thereby realizing the generation and display of the target special effect. Specifically, for example, through the demand information, the target special effect category of the target special effect is determined, such as "background special effect", "headwear special effect", "beauty special effect", etc. The specific implementation method can be implemented based on the first prompt word. For the specific implementation method of determining the target special effect category based on the first prompt word, please refer to Figure 4 The relevant introduction in the illustrated embodiment, of course, in other implementations, can also be directly mapped to the corresponding special effects template based on the demand information, can be set based on needs, and will not be repeated here. Afterwards, based on the target special effects category, the corresponding special effects template is obtained, and the special effects material is processed through the special effects template, including size conversion, cropping, etc. For example, when the special effects category corresponding to the special effects template is "background special effects", the special effects material is cropped so that it does not overlap with the objects in the target image, and the corresponding special effects rendering data is generated. The special effects rendering data is rendered to generate the target special effects in the target image.

[0090] Further, exemplarily, the special effect template includes at least one special effect field, and the special effect field is used to add at least one special effect element to the target image, such as Fig.11 As shown, the specific implementation of step S205 includes:

[0091] Step S2051: Map the target special effect material to the corresponding target special effect field in the special effect template to obtain the target special effect template.

[0092] Step S2052: Process the target special effect template using a rendering engine to generate a target special effect in the target image.

[0093] Exemplarily, the special effect field in the special effect template is used to add at least one special effect element to the target image. For example, the special effect template includes field #1, field #2 and field #3, wherein field #1 corresponds to "headwear special effect", which is used to add a virtual headwear (special effect element, the same below) to the character in the target image; field #2 corresponds to "beauty special effect", which is used to add beauty effect to the face of the character in the target image; field #3 corresponds to "background special effect", which is used to add a virtual background to the character in the target image. The target special effect material is mapped to the corresponding target special effect field in the special effect template. For example, the first special effect material generated based on user-generated content in the above step is mapped to field #1; the two (two groups) of second special effect materials generated based on the model in the above step are mapped to field #2 and field #3 respectively. Thus, the purpose of adding "headwear special effect", "beauty special effect" and "background special effect" to the target image is achieved at the same time. Afterwards, the configured target special effect template is processed by the rendering engine to achieve the purpose of generating the target image in the target image. The specific implementation principle of image rendering is existing technology and will not be repeated here.

[0094] Among them, the above-mentioned special effects template can be manually configured by the user, or it can be automatically generated based on demand information. The specific implementation method can be achieved through a pre-trained model, which will not be repeated here.

[0095] In the steps of this embodiment, by configuring the special effects template, the effect of adding multiple special effects elements at one time can be achieved, and the special effects material corresponding to each special effects element is independently acquired based on the previous steps, and therefore has better matching and diversity with the image content. At the same time, by mapping different types of special effects materials (first special effects material and second special effects material) through the special effects template, the user's personalized configuration needs can be met.

[0096] Furthermore, the special effect field corresponds to at least one parameter field, and the parameter field is used to represent the rules for processing the special effect material corresponding to the special effect field to generate the corresponding special effect element. Taking the #1 field in the above example as an example, illustratively, the #1 field corresponds to 3 parameter fields, namely the a parameter field, the b parameter field and the c parameter field, wherein the a parameter field represents the position of the "headwear special effect" in the image, more specifically, for example, the distance from the head outline of the character in the target image; the b parameter field represents the color of the "headwear special effect"; and the c parameter field represents the transparency of the "headwear special effect".

[0097] In this embodiment, by further configuring one or more parameter fields corresponding to the special effect fields in the special effect template, refined control of the visual effect of the target special effect is achieved, further improving the visual performance and diversity of the target special effect, improving the generation efficiency of the target special effect, and the application effect in the image.

[0098] In this embodiment, the implementation method of step S201 is the same as that of the present disclosure. Figure 2 The implementation method of step S101 in the illustrated embodiment is the same and will not be described in detail here.

[0099] Corresponding to the special effect generation method of the above embodiment, Fig.12 This is a structural block diagram of a special effect generation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown.

[0100] Reference Fig.12 , the special effect generating device 3 comprises:

[0101] An interactive module 31 is used to receive demand information input by a user, where the demand information is used to describe the generation requirements of a target special effect based on a natural language;

[0102] A generating module 32, for generating special effect materials matching the image content of the currently displayed target image according to the demand information;

[0103] The rendering module 33 is used to generate a target special effect in a target image based on the special effect material.

[0104] In one embodiment of the present disclosure, the interaction module 31 is further used to: display at least one special effect material in the first interface; and in response to a selection operation of a target special effect material in the at least one special effect material, display a target special effect corresponding to the target special effect material in the target image.

[0105] In one embodiment of the present disclosure, the generation module 32 is specifically used to: generate a first prompt word based on demand information, the first prompt word is used to characterize the target special effect category to which the target special effect belongs; obtain a data index based on the first prompt word, the data index is used to indicate a user-generated content library corresponding to the target special effect category; obtain corresponding special effect material based on the data index and the image content of the target image.

[0106] In one embodiment of the present disclosure, the generation module 32 is further used to: obtain a first rendering parameter based on a first prompt word, and the first rendering parameter is used to characterize the position and / or distribution of a target special effect generated based on the special effects material in a target image; the rendering module 33 is specifically used to: generate a target special effect in a target image according to the special effects material and the first rendering parameter.

[0107] In one embodiment of the present disclosure, the generation module 32 is specifically used to: generate a second prompt word according to the demand information, the second prompt word is used to guide the target model to generate special effect materials that meet the generation requirements; generate special effect materials based on the second prompt word and the target model.

[0108] In one embodiment of the present disclosure, when generating the second prompt word according to the demand information, the generation module 32 is specifically used to: obtain the image content features of the currently displayed target image; and generate the second prompt word based on the image content features and the demand information.

[0109] In one embodiment of the present disclosure, when the generation module 32 generates the second prompt word based on the image content features and the demand information, it is specifically used to: generate a first text parameter based on the image content features, the first text parameter is used to describe the content semantics of the target image; based on the demand information, in combination with the prompt word template, generate a prompt word text, the prompt word text includes input parameters, the prompt word text representation is processed for the input parameters to generate the processing logic of the special effects material that meets the generation requirements; the first text parameter is used as the input parameter, the prompt word text is inserted into the prompt word text, and the second prompt word is generated.

[0110] In one embodiment of the present disclosure, the generation module 32 is specifically used to: generate a first special effects material and a second special effects material according to demand information, wherein the first special effects material is pre-generated user-generated content and the second special effects material is real-time generated model-generated content; generate special effects material according to a combination of the first special effects material and the second special effects material.

[0111] In one embodiment of the present disclosure, the rendering module 33 is specifically used to: obtain a special effect template corresponding to a target special effect, wherein the special effect template is used to process at least one special effect material; perform image rendering based on the special effect template and the special effect material to generate a target special effect in a target image.

[0112] In one embodiment of the present disclosure, the special effects template includes at least one special effects field, and the special effects field is used to add at least one special effects element to the target image; when the rendering module 33 performs image rendering based on the special effects template and the special effects material and generates a target special effect in the target image, it is specifically used to: map the target special effects material to the corresponding target special effects field in the special effects template to obtain the target special effects template; use the rendering engine to process the target special effects template to generate the target special effects in the target image.

[0113] In one embodiment of the present disclosure, the special effect field corresponds to at least one parameter field, and the parameter field is used to represent a rule for processing the special effect material corresponding to the special effect field to generate a corresponding special effect element.

[0114] The interaction module 31, the generation module 32 and the rendering module 33 are connected in sequence. The special effect generation device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0115] Fig.13 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Fig.13 As shown, the electronic device 4 includes:

[0116] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0117] The memory 42 stores computer executable instructions;

[0118] The processor 41 executes the computer execution instructions stored in the memory 42 to implement the following Figure 2-Figure 11 The special effects generating method in the illustrated embodiment.

[0119] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0120] For related instructions, please refer to Figure 2-Figure 11 The relevant descriptions and effects corresponding to the steps in the corresponding embodiments can be understood, and no further elaboration is made here.

[0121] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the present invention when executed by a processor. Figure 2-Figure 11 The special effects generation method provided in any one of the corresponding embodiments.

[0122] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0123] refer to Fig.14 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure, and the electronic device 900 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.14 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0124] like Fig.14 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0125] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.14 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0126] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0127] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0128] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0129] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0130] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0133] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0134] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] In a first aspect, according to one or more embodiments of the present disclosure, a special effect generation method is provided, comprising:

[0136] Receive demand information input by a user, wherein the demand information is used to describe the generation requirements of a target special effect based on natural language; generate special effect materials that match the image content of a currently displayed target image based on the demand information; and generate the target special effect in the target image based on the special effect materials.

[0137] According to one or more embodiments of the present disclosure, the method further includes: displaying at least one of the special effects materials in the first interface; and in response to a selection operation of a target special effects material in the at least one special effects material, displaying a target special effect corresponding to the target special effects material in the target image.

[0138] According to one or more embodiments of the present disclosure, generating special effects material that matches the image content of the currently displayed target image based on the demand information includes: generating a first prompt word based on the demand information, the first prompt word being used to characterize a target special effects category to which the target special effects belong; obtaining a data index based on the first prompt word, the data index being used to indicate a user-generated content library corresponding to the target special effects category; and obtaining corresponding special effects material based on the data index and the image content of the target image.

[0139] According to one or more embodiments of the present disclosure, the method further includes: obtaining a first rendering parameter based on the first prompt word, the first rendering parameter being used to characterize the position and / or distribution of a target special effect generated based on the special effects material in a target image; generating the target special effect in the target image based on the special effects material, including: generating the target special effect in the target image according to the special effects material and the first rendering parameter.

[0140] According to one or more embodiments of the present disclosure, generating special effects material that matches the image content of the currently displayed target image based on the demand information includes: generating a second prompt word based on the demand information, the second prompt word being used to guide the target model to generate special effects material that meets the generation requirements; and generating the special effects material based on the second prompt word and the target model.

[0141] According to one or more embodiments of the present disclosure, generating a second prompt word according to the demand information includes: acquiring image content features of a currently displayed target image; and generating the second prompt word based on the image content features and the demand information.

[0142] According to one or more embodiments of the present disclosure, the generating of the second prompt word based on the image content feature and the requirement information includes: generating a first text parameter based on the image content feature, the first text parameter being used to describe the content semantics of the target image; generating a prompt word text based on the requirement information and in combination with a prompt word template, the prompt word text including input parameters, the prompt word text representing processing logic for processing the input parameters to generate special effects material that meets the generation requirements; and using the first text parameter as an input parameter and inserting it into the prompt word text to generate the second prompt word.

[0143] According to one or more embodiments of the present disclosure, generating special effects material that matches the image content of the currently displayed target image according to the demand information includes: generating a first special effects material and a second special effects material according to the demand information, wherein the first special effects material is pre-generated user-generated content and the second special effects material is real-time generated model-generated content; generating special effects material according to a combination of the first special effects material and the second special effects material.

[0144] According to one or more embodiments of the present disclosure, generating the target special effect in the target image based on the special effects material includes: obtaining a special effects template corresponding to the target special effect, wherein the special effects template is used to process at least one special effects material; performing image rendering based on the special effects template and the special effects material, and generating the target special effect in the target image.

[0145] According to one or more embodiments of the present disclosure, the special effects template includes at least one special effects field, and the special effects field is used to add at least one special effects element to the target image; the image rendering is performed based on the special effects template and the special effects material to generate the target special effect in the target image, including: mapping the target special effects material to the corresponding target special effects field in the special effects template to obtain a target special effects template; and processing the target special effects template using a rendering engine to generate the target special effect in the target image.

[0146] According to one or more embodiments of the present disclosure, the special effect field corresponds to at least one parameter field, and the parameter field is used to represent a rule for processing the special effect material corresponding to the special effect field to generate a corresponding special effect element.

[0147] In a second aspect, according to one or more embodiments of the present disclosure, a special effect generation device is provided, comprising:

[0148] An interactive module, used to receive demand information input by a user, where the demand information is used to describe the generation requirements of a target special effect based on a natural language;

[0149] A generation module, used to generate special effect materials matching the image content of the currently displayed target image according to the demand information;

[0150] A rendering module is used to generate the target special effect in the target image based on the special effect material.

[0151] According to one or more embodiments of the present disclosure, the interaction module is further used to: display at least one of the special effects materials in the first interface; and in response to a selection operation of a target special effects material in the at least one special effects material, display a target special effect corresponding to the target special effects material in the target image.

[0152] According to one or more embodiments of the present disclosure, the generation module is specifically used to: generate a first prompt word according to the demand information, the first prompt word is used to characterize the target special effect category to which the target special effect belongs; obtain a data index according to the first prompt word, the data index is used to indicate a user-generated content library corresponding to the target special effect category; obtain corresponding special effect material according to the data index and the image content of the target image.

[0153] According to one or more embodiments of the present disclosure, the generation module is further used to: obtain a first rendering parameter based on the first prompt word, and the first rendering parameter is used to characterize the position and / or distribution of the target special effect generated based on the special effect material in the target image; the rendering module 33 is specifically used to: generate the target special effect in the target image according to the special effect material and the first rendering parameter.

[0154] According to one or more embodiments of the present disclosure, the generation module is specifically used to: generate a second prompt word according to the demand information, the second prompt word is used to guide the target model to generate special effects material that meets the generation requirements; generate the special effects material based on the second prompt word and the target model.

[0155] According to one or more embodiments of the present disclosure, when the generation module generates the second prompt word according to the demand information, it is specifically used to: obtain image content features of the currently displayed target image; and generate the second prompt word based on the image content features and the demand information.

[0156] According to one or more embodiments of the present disclosure, when the generation module generates the second prompt word based on the image content feature and the requirement information, it is specifically used to: generate a first text parameter based on the image content feature, and the first text parameter is used to describe the content semantics of the target image; based on the requirement information, in combination with a prompt word template, generate a prompt word text, the prompt word text includes input parameters, and the prompt word text represents processing logic for processing the input parameters to generate special effects materials that meet the generation requirements; use the first text parameter as an input parameter, insert the prompt word text into the prompt word text, and generate the second prompt word.

[0157] According to one or more embodiments of the present disclosure, the generation module is specifically used to: generate a first special effects material and a second special effects material according to the demand information, wherein the first special effects material is pre-generated user-generated content and the second special effects material is real-time generated model-generated content; generate special effects material according to a combination of the first special effects material and the second special effects material.

[0158] According to one or more embodiments of the present disclosure, the rendering module is specifically used to: obtain a special effect template corresponding to the target special effect, wherein the special effect template is used to process at least one special effect material; perform image rendering based on the special effect template and the special effect material, and generate the target special effect in the target image.

[0159] According to one or more embodiments of the present disclosure, the special effects template includes at least one special effects field, and the special effects field is used to add at least one special effects element to the target image; when the rendering module 33 performs image rendering based on the special effects template and the special effects material and generates the target special effect in the target image, it is specifically used to: map the target special effects material to the corresponding target special effects field in the special effects template to obtain a target special effects template; use a rendering engine to process the target special effects template to generate the target special effect in the target image.

[0160] According to one or more embodiments of the present disclosure, the special effect field corresponds to at least one parameter field, and the parameter field is used to represent a rule for processing the special effect material corresponding to the special effect field to generate a corresponding special effect element.

[0161] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0162] The memory stores computer-executable instructions;

[0163] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the special effects generating method described in the first aspect and various possible designs of the first aspect.

[0164] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the special effects generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0165] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the special effects generation method described in the first aspect and various possible designs of the first aspect.

[0166] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0167] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0168] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A special effect generation method, characterized in that: include: Receive demand information input by a user, where the demand information is used to describe generation requirements of a target special effect based on a natural language; Generate special effect materials that match the image content of the currently displayed target image according to the demand information; Based on the special effect material, the target special effect is generated in the target image.

2. The method according to claim 1, characterized in that: The method further comprises: In the first interface, display at least one of the special effect materials; In response to a selection operation of a target special effect material from the at least one special effect material, a target special effect corresponding to the target special effect material is displayed in the target image.

3. The method according to claim 1, characterized in that The step of generating, according to the demand information, special effect materials matching the image content of the currently displayed target image comprises: Generate a first prompt word according to the demand information, where the first prompt word is used to represent the target special effect category to which the target special effect belongs; According to the first prompt word, a data index is obtained, where the data index is used to indicate a user-generated content library corresponding to the target special effect category; According to the data index and the image content of the target image, corresponding special effect materials are obtained.

4. The method according to claim 3, characterized in that The method further comprises: Based on the first prompt word, obtaining a first rendering parameter, where the first rendering parameter is used to characterize a position and / or distribution of a target special effect generated based on the special effect material in a target image; Generating the target special effect in the target image based on the special effect material includes: The target special effect is generated in the target image according to the special effect material and the first rendering parameter.

5. The method according to claim 1, characterized in that The step of generating, according to the demand information, special effect materials matching the image content of the currently displayed target image comprises: Generate a second prompt word according to the demand information, where the second prompt word is used to guide the target model to generate special effect materials that meet the generation requirements; The special effect material is generated based on the second prompt word and the target model.

6. The method according to claim 5, characterized in that The step of generating a second prompt word according to the demand information includes: Obtain image content features of the currently displayed target image; The second prompt word is generated based on the image content feature and the demand information.

7. The method according to claim 6, characterized in that The generating the second prompt word based on the image content feature and the demand information includes: Based on the image content feature, generating a first text parameter, wherein the first text parameter is used to describe the content semantics of the target image; Based on the demand information and in combination with the prompt word template, a prompt word text is generated, wherein the prompt word text includes input parameters, and the prompt word text represents processing logic for processing the input parameters to generate special effect materials that meet the generation requirements; The first text parameter is used as an input parameter, and the prompt word text is inserted to generate the second prompt word.

8. The method according to claim 1, characterized in that The step of generating, according to the demand information, special effect materials matching the image content of the currently displayed target image comprises: Generate a first special effect material and a second special effect material according to the demand information, wherein the first special effect material is pre-generated user-generated content, and the second special effect material is real-time generated model-generated content; Special effect material is generated according to the combination of the first special effect material and the second special effect material.

9. The method according to claim 1, characterized in that: The step of generating the target special effect in the target image based on the special effect material includes: Acquire a special effect template corresponding to the target special effect, wherein the special effect template is used to process at least one special effect material; Image rendering is performed based on the special effect template and the special effect material to generate the target special effect in the target image.

10. The method according to claim 9, characterized in that The special effect template includes at least one special effect field, and the special effect field is used to add at least one special effect element to the target image; The performing image rendering based on the special effect template and the special effect material to generate the target special effect in the target image includes: Mapping the target special effect material to the corresponding target special effect field in the special effect template to obtain the target special effect template; The target special effect template is processed by using a rendering engine to generate the target special effect in the target image.

11. The method according to claim 10, characterized in that The special effect field corresponds to at least one parameter field, and the parameter field is used to represent a rule for processing the special effect material corresponding to the special effect field to generate a corresponding special effect element.

12. A special effect generating device, characterized in that: include: An interactive module, used to receive demand information input by a user, where the demand information is used to describe the generation requirements of a target special effect based on a natural language; A generation module, used to generate special effect materials matching the image content of the currently displayed target image according to the demand information; A rendering module is used to generate the target special effect in the target image based on the special effect material.

13. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the special effect generation method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the special effect generation method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the special effect generating method according to any one of claims 1 to 11 is implemented.

Images