Multimedia editing method and device, equipment and medium

Abstract

The embodiment of the invention relates to a multimedia editing method and device, equipment and a medium, and the method comprises the steps: responding to a material filling operation for a target multimedia template, obtaining a filling material corresponding to the material filling operation, and generating first multimedia content based on the target multimedia template and the filling material; the target multimedia template comprises editing operation information corresponding to the to-be-filled material; in response to a received editing request for a target filling material in the filling materials, displaying an editing function corresponding to the target filling material; the editing function at least comprises a content generation function; under the condition that the content generation function is triggered, obtaining a target generation model corresponding to the target filling material, and generating a target generation material corresponding to the target filling material by using the target generation model; and replacing the target filling material in the first multimedia content with the target generation material to obtain second multimedia content. According to the embodiment of the invention, the multimedia editing effect can be further enriched.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a multimedia editing method, apparatus, device, and medium. Background Art

[0002] In order to enable users to perform multimedia editing conveniently and quickly, some multimedia editing software or websites can provide multimedia templates for users. Users can fill in their own materials based on the multimedia templates, so as to conveniently and quickly obtain multimedia content such as videos required by the users. However, the inventors have found through research that the existing method of editing using multimedia templates still needs to be improved. Summary of the Invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a multimedia editing method, apparatus, device, and medium.

[0004] An embodiment of the present disclosure provides a multimedia editing method, the method including: in response to a material filling operation for a target multimedia template, obtaining filling materials corresponding to the material filling operation, so as to generate first multimedia content based on the target multimedia template and the filling materials; wherein the target multimedia template includes editing operation information corresponding to the materials to be filled; in response to receiving an editing request for a target filling material in the filling materials, displaying an editing function corresponding to the target filling material; the editing function at least includes a content generation function; in a case where the content generation function is triggered, obtaining a target generation model corresponding to the target filling material, and using the target generation model to generate target generation materials corresponding to the target filling material; and replacing the target filling material in the first multimedia content with the target generation materials, so as to obtain second multimedia content.

[0005] Optionally, the obtaining the target generation model corresponding to the target filling material includes: displaying selection items corresponding to a plurality of generation models; and obtaining the target generation model corresponding to the target filling material based on a triggered selection item among the selection items corresponding to the plurality of generation models.

[0006] Optionally, the displaying selection items corresponding to a plurality of generation models includes: displaying selection items of multiple content generation categories; wherein each content generation category corresponds to a plurality of generation models, and the similarity between generation models in the same content generation category is greater than the similarity between generation models in different content generation categories; and in response to a selection item of a target content generation category among the selection items of the multiple content generation categories being triggered, displaying selection items of a plurality of generation models corresponding to the target content generation category.

[0007] Optionally, the option corresponding to the generation model is presented in the form of a generation effect example corresponding to the generation model.

[0008] Optionally, generating the target generated material corresponding to the target filling material by using the target generation model includes: generating a plurality of generated materials corresponding to the target filling material by using the target generation model; wherein different generated materials are generated by the target generation model using different model parameters; displaying the plurality of generated materials, and determining the target generated material corresponding to the target filling material based on the triggered generated material.

[0009] Optionally, determining the target generated material corresponding to the target filling material based on the triggered generated material includes: in response to receiving an adjustment request for the triggered generated material, obtaining adjustment information corresponding to the adjustment request; the adjustment information includes an updated model prompt and / or updated model processing parameters; based on the adjustment information, generating the target generated material corresponding to the target filling material by using the target generation model corresponding to the triggered generated material.

[0010] Optionally, obtaining the adjustment information corresponding to the adjustment request includes: displaying the current prompt corresponding to the triggered generated material and the current model processing parameters; wherein the triggered generated material is generated by the target generation model processing the target filling material based on the current prompt and the current model processing parameters; providing a first modification control corresponding to the current prompt and a second modification control corresponding to the current model processing parameters; and obtaining the adjustment information corresponding to the adjustment request based on the operation result corresponding to the first modification control and the operation result corresponding to the second modification control.

[0011] Optionally, displaying the plurality of generated materials includes: displaying the target filling material and the plurality of generated materials side by side or in parallel, and the display position of the target filling material is before the display positions of the plurality of generated materials.

[0012] Optionally, the method further includes: generating a new multimedia template based on the second multimedia content; wherein the new multimedia template records the editing operation information corresponding to the replaceable target generated material, and the editing operation information of the target generated material includes the information of editing the target generated material by using the target generation model.

[0013] Optionally, the generation model includes one or more of an image generation model, an audio generation model, and a video generation model.

[0014] An embodiment of the present disclosure also provides a multimedia editing device, including: a first generation module, configured to obtain filling materials corresponding to the material filling operation in response to a material filling operation for a target multimedia template, so as to generate first multimedia content based on the target multimedia template and the filling materials; wherein, the target multimedia template includes editing operation information corresponding to the materials to be filled; a function display module, configured to display an editing function corresponding to the target filling material in response to receiving an editing request for the target filling material in the filling materials; the editing function at least includes a content generation function; a model generation module, configured to obtain a target generation model corresponding to the target filling material and generate target generation materials corresponding to the target filling material by using the target generation model when the content generation function is triggered; a second generation module, configured to replace the target filling material in the first multimedia content with the target generation material, so as to generate second multimedia content based on the target multimedia template and the target generation materials.

[0015] An embodiment of the present disclosure also provides an electronic device, where the electronic device includes: a processor; a memory for storing executable instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the multimedia editing method provided by the embodiment of the present disclosure.

[0016] An embodiment of the present disclosure also provides a computer-readable storage medium, where the storage medium stores a computer program, and the computer program is used to execute the multimedia editing method provided by the embodiment of the present disclosure.

[0017] The above technical solution provided by the embodiment of the present disclosure can, in response to a material filling operation for a target multimedia template, obtain filling materials corresponding to the material filling operation, generate first multimedia content based on the target multimedia template and the filling materials, and can also display an editing function (at least including a content generation function) corresponding to the target filling material when receiving an editing request for the target filling material in the filling materials. When the content generation function is triggered, obtain a target generation model corresponding to the target filling material, and generate target generation materials corresponding to the target filling material by using the target generation model; replace the target filling material in the first multimedia content with the target generation material to obtain second multimedia content. The above method can provide a content generation function for the filling materials of the target multimedia template and generate corresponding materials by using the generation model, which helps to further enrich the multimedia editing effect.

[0018] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a schematic flowchart of a multimedia editing method provided by an embodiment of the present disclosure;

[0022] Figure 2 It is a schematic diagram of a multimedia editing interface provided by an embodiment of the present disclosure;

[0023] Figure 3 It is a schematic diagram of a multimedia editing interface provided by an embodiment of the present disclosure;

[0024] Figure 4 It is a schematic diagram of a multimedia editing interface provided by an embodiment of the present disclosure;

[0025] Figure 5 It is a schematic structural diagram of a multimedia editing device provided by an embodiment of the present disclosure;

[0026] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In order to more clearly understand the above objects, features, and advantages of the present disclosure, the following will further describe the solutions of the present disclosure. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0028] In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0029] Figure 1 It is a schematic flowchart of a multimedia editing method provided by an embodiment of the present disclosure. This method can be executed by a multimedia editing device, where the device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 1 shown, this method mainly includes the following steps S102 to step S108:

[0030] Step S102: In response to a material filling operation for a target multimedia template, obtain the filling materials corresponding to the material filling operation, so as to generate a first multimedia content based on the target multimedia template and the filling materials.

[0031] In practical applications, the display effects of multiple candidate multimedia templates can be presented to the user on the template selection interface, so that the user can select the required target multimedia template from multiple candidate multimedia templates. It is also possible to display the video edited by the target multimedia template in the first application software (such as a short video playback software), and set the acquisition method of the target multimedia template, so that the user can directly obtain the target multimedia template or jump to the second application software (such as multimedia editing software) to obtain the target multimedia template. The target multimedia template can be a video template or an image template, etc., which is not limited here. Specifically, the target multimedia template contains the editing operation information corresponding to the materials to be filled. In addition, if there are other materials in the target multimedia template, it will also contain the editing operation information of other materials. The editing operation information is used to indicate the editing operations to be performed on the materials. The materials can be one or more of images, videos, audios, texts, stickers, special effects, etc., which is not limited here. In some implementation examples, the target multimedia template contains sample materials, and at least some of the sample materials are set with replaceable identifiers, which can be regarded as corresponding to the aforementioned materials to be filled, and the user can replace such replaceable materials with the materials uploaded by himself. In other implementation examples, the target multimedia template contains sample materials, and at least some of the sample materials are deleted, which can be regarded as the aforementioned materials to be filled, and the user can directly fill in the materials uploaded by himself at the corresponding positions.

[0032] Exemplarily, an editing interface for the target multimedia template can be provided to the user, and the slots for the materials to be filled are presented in the editing interface. The number of slots depends on the number of materials to be filled. The user can upload the corresponding filling materials for each slot respectively. The electronic device executing the multimedia editing method can generate a first multimedia content based on the target multimedia template and the filling materials. Specifically, if the original replaceable materials in the target multimedia template are deleted, the filling materials can be directly filled into the corresponding positions of the target multimedia template. If the original replaceable materials in the target multimedia template are not deleted, the corresponding replaceable materials in the target multimedia template can be replaced with the filling materials to obtain the first multimedia content. Among them, the first multimedia content can be a draft file, which contains both materials and the editing operation information corresponding to each material.

[0033] Step S104: In response to receiving an editing request for a target filling material in the filling materials, display the editing functions corresponding to the target filling material; the editing functions at least include a content generation function.

[0034] In practical applications, the embodiments of the present disclosure provide an editing function for target filling materials. Users can further edit the filling materials according to their needs. The editing function at least includes a content generation function, and may also include other functions such as cropping and replacement, which are not limited herein.

[0035] For ease of understanding, reference may be made to Figure 2 a schematic diagram of a multimedia editing interface as shown. A total of 3 filling materials uploaded by the user are shown, and it is indicated that the user currently triggers the second filling material among the 3 filling materials. At this time, the second filling material triggered by the user is directly displayed in the preview area of the editing interface. In practical applications, the triggered material can be highlighted (such as changing the background color, adding a highlighted border, etc.), Figure 2 simply indicated by a bold black border in the figure. Further, Figure 2 multiple editable functions that are currently available are indicated, such as an AI (Artificial Intelligence) editing function (corresponding to the aforementioned content generation function), a replacement function, a cropping function, etc. Figure 2 This is only for illustrative purposes and may also include other functions, which are not limited herein.

[0036] Step S106, when the content generation function is triggered, obtain the target generation model corresponding to the target filling material, and use the target generation model to generate the target generation material corresponding to the target filling material.

[0037] In practical applications, the target generation model corresponding to the target filling material can be a preset default model or a generation model selected from multiple generation models. Specifically, the user can select the target generation model according to their preferences, or it can be a target generation model automatically matched with the style or type of the target filling material through a preset algorithm or a preset mapping relationship. The embodiments of the present disclosure do not limit the determination method of the target generation model.

[0038] The target generation model can generate new materials (target generated materials) with reference to the target filling materials. In practical applications, the target generation model includes one or more of an image generation model, an audio generation model, and a video generation model. Taking the target generation model as an image generation model as an example, exemplarily, the target filling material is an image, and this image generation model can be an image-to-image model. That is, the image-to-image model can draw a new image based on the target filling material. In addition, in practical applications, the target generation model can also be an image-to-video model (that is, generating a video based on the input image), a video-to-video model (that is, generating a new video based on the input video), etc., which are not limited here. Compared with directly performing simple edits such as conventional cropping, adding stickers, or adjusting the color tone on the target filling material, the method of obtaining the target generated material through the generation model is more flexible, and the resulting target generated material has a richer and more appealing form of expression, which can better meet the user's needs.

[0039] Step S108, replace the target filling material in the first multimedia content with the target generated material to obtain the second multimedia content.

[0040] As mentioned above, through the target generation model, target filling materials with richer forms of expression can be generated. Replacing the target filling material in the first multimedia content with the target generated material can obtain the second multimedia content. The second multimedia content can be only a draft file, or can be exported as multimedia data such as a video. In addition, a new multimedia template can be further generated based on the second multimedia content. Compared with the original target multimedia template, the new multimedia template adds editing operation information for content generation using the target generation model for the material to be filled. Specifically, the format and use of the second multimedia content can be flexibly set according to requirements, which are not limited here.

[0041] In summary, the above multimedia editing method provided by the embodiments of the present disclosure can provide a content generation function for the filling materials of the target multimedia template and use the generation model to generate corresponding materials, which helps to further enrich the multimedia editing effect.

[0042] In some embodiments, the target generation model can be selected by the user according to needs. The above step S106, that is, the step of obtaining the target generation model corresponding to the target filling material, can be executed with reference to the following steps a and b:

[0043] Step a, display selection items corresponding to multiple generation models. In some embodiments, the selection items of multiple generation models can be directly presented. In other embodiments, the generation models can be classified to make it more convenient for users to efficiently select the required generation model. On this basis, exemplarily, step a can be executed with reference to the following steps a1 and a2:

[0044] Step a1, displaying multiple options for content generation categories; wherein each content generation category corresponds to multiple generation models, and illustratively, the similarity between generation models in the same content generation category is greater than the similarity between generation models in different content generation categories. The disclosed embodiment does not limit the evaluation dimension of similarity, such as the evaluation dimension of similarity can be determined based on the content generation category. In practical applications, the content generation categories can be divided based on preset dimensions, such as the content generation categories can be divided according to realistic style, cartoon style and other styles, or according to scenes such as scenery and streets. The division method of the content generation categories is not limited here. For each content generation category, the similarity of the generation models contained therein under the evaluation dimension corresponding to the content generation category is generally high, such as the generation models contained in the content generation category used to characterize the cartoon style have good cartoon content generation capabilities and can generate cartoon materials.

[0045] In actual applications, the embodiments of the present disclosure do not limit the presentation form of the content generation category selection items. For example, the corresponding selection items may be presented directly using the category name of the content generation category, or the selection items of the content generation category may be presented in the form of effect images and / or text descriptions, so that the user can clearly understand the difference between the various content generation categories and select the desired content generation category from them.

[0046] Step a2, in response to the target content generation category selection item among the multiple content generation category selection items being triggered, display multiple generation model selection items corresponding to the target content generation category. Exemplarily, the selection items corresponding to the generation model are presented in the form of generation effect samples corresponding to the generation model, so that the user can clearly know the generation effects that can be achieved by each generation model.

[0047] Step b: based on the triggered selection item among the selection items corresponding to the multiple generation models, obtain the target generation model corresponding to the target filling material. Exemplarily, when the selection item corresponding to the generation model is presented in the form of a generation effect sample corresponding to the generation model, the embodiment of the present disclosure can respond to the selection operation on the generation effect sample and use the generation model corresponding to the selected generation effect sample as the target generation model.

[0048] For easier understanding, please refer to Figure 3 shows a schematic diagram of a multimedia editing interface, indicating that the user has selected content generation category 1 (in bold) from content generation categories 1 to 3 as the desired target content generation category, and exemplarily displays AI model selection items 1 to AI model selection items 3 included in content generation category 1, where the AI ​​model is the aforementioned generation model. Figure 3 ​Among them, sample generation effects corresponding to each AI model are respectively shown, so that users can clearly and intuitively know the effects that can be achieved by each AI model based on the sample generation effects of each model, so as to quickly select the required AI model. In practical applications, the sample generation effects corresponding to each AI model can be presented in the form of cards, and each card is associated with the corresponding AI model, the prompt words corresponding to the AI model, and information such as model processing parameters (such as processing intensity). If a certain card is triggered, it is determined that the user selects the AI model corresponding to the card as the target generation model.

[0049] After determining the target generation model, the target generation model can be used to generate the target generation material corresponding to the target filling material. The embodiments of the present disclosure provide specific implementation examples of using the target generation model to generate the target generation material corresponding to the target filling material in step S106 above, which can be executed with reference to the following steps A and B:

[0050] Step A, using the target generation model to generate multiple generation materials corresponding to the target filling material; among them, different generation materials are generated by the target generation model using different model parameters. For example, the target generation model can fine-tune some model parameters to generate multiple generation materials with different effects. The styles of the multiple generation materials are similar, but there may be certain differences in aspects such as the presentation form.

[0051] Step B, displaying the multiple generation materials, and determining the target generation material corresponding to the target filling material based on the triggered generation material.

[0052] In the above manner, more freedom of choice can be provided for users, and users can select the generation material that best meets their own needs from them.

[0053] In order to ensure the user experience and facilitate users to intuitively select the required generation material from multiple generation materials, when displaying the multiple generation materials in step B, the target filling material and the multiple generation materials can be displayed side by side or in parallel, and the display position of the target filling material is before the display position of the multiple generation materials. Exemplarily, a preview area and a material display area are presented in the multimedia editing interface. The target filling material and the multiple generation materials can be located in the material display area, and the selected material can be enlarged and displayed in the preview area.

[0054] In order to make the finally obtained target generation material better meet the user's needs, when determining the target generation material corresponding to the target filling material based on the triggered generation material in step B, the following steps 1 and 2 can also be executed with reference to:

[0055] Step 1, in response to receiving an adjustment request for the triggered generated material, obtain the adjustment information corresponding to the adjustment request; the adjustment information includes the updated model prompt and / or the updated model processing parameters. In practical applications, users can further adjust the materials generated by the target generation model according to their needs. For example, adjust the prompt of the target generation model and / or model processing parameters such as intensity. In some specific implementation examples, the steps of obtaining the adjustment information corresponding to the adjustment request can be executed with reference to the following steps 1.1 to 1.3:

[0056] Step 1.1, display the current prompt corresponding to the triggered generated material and the current model processing parameters; wherein, the triggered generated material is generated by the target generation model based on the current prompt and the current model processing parameters for processing the target filling material.

[0057] When the user initiates an adjustment request, the current prompt corresponding to the triggered generated material and the current model processing parameters can be displayed to the user. The current model processing parameters such as the intensity parameter used by the model are parameters that can be adjusted by the user.

[0058] Step 1.2, provide a first modification control corresponding to the current prompt and a second modification control corresponding to the current model processing parameters; in practical applications, different first modification controls and second modification controls can be provided simultaneously, or the first modification control and the second modification control can be provided successively, which is not limited here.

[0059] Step 1.3, based on the operation result corresponding to the first modification control and the operation result corresponding to the second modification control, obtain the adjustment information corresponding to the adjustment request. Specifically, the user can modify the current prompt to obtain the updated prompt. In practical applications, the user can modify some words in the current prompt (such as changing black hair to brown hair, etc.), or completely modify the current prompt, which can be flexibly set specifically. The user can also modify the current model processing parameters to obtain the updated processing parameters. For example, modify the parameter of the model processing intensity. The user can operate the first modification control and / or the second modification control according to the needs to set a new current prompt or a new model processing parameter. Among them, the operation result corresponding to the unoperated modification control can be regarded as empty.

[0060] Step 2, based on the adjustment information, use the target generation model corresponding to the triggered generated material to generate the target generated material corresponding to the target filling material. Based on the updated model prompt and / or the updated model processing parameters, make the target generation model regenerate the target generated material corresponding to the target filling material, so as to obtain the material that meets the user's needs.

[0061] For ease of understanding, reference can be made to Figure 4A schematic diagram of a multimedia editing interface is shown. The user can select from 4 different AI materials (corresponding to the aforementioned generated materials) generated by the currently selected target AI model (corresponding to the aforementioned target generation model). Assuming that AI material 1 is selected, at this time, AI material 1 can be enlarged and displayed in the preview area. The user can directly use AI material 1 as the final target generation material. In addition, the user can also initiate an adjustment request for AI material 1 by triggering the adjustment control, so as to further set new model prompt words and / or new model processing parameters, so that the target AI model can regenerate the target generation material required by the user based on the new model prompt words and / or new model processing parameters.

[0062] Furthermore, the multimedia editing method provided by the embodiments of the present disclosure further includes: generating a new multimedia template based on the second multimedia content; wherein, the new multimedia template records the editing operation information corresponding to the replaceable target generation material, and the editing operation information of the target generation material includes the information of performing an editing operation on the target generation material using the target generation model. Specifically, compared with the original target multimedia template, the new multimedia template adds the editing operation information of generating content for the target generation material using the target generation model, or it can also be said that the editing operation information corresponding to the target generation material replaces the editing operation information used for the target filling material in the target multimedia template. In specific implementation, if the type of the target filling material in the target multimedia template is a non-content generation type, such as the target filling material is an original picture or video taken by the user, etc., then the editing information of the target generation model used for the target generation material can be added to the new multimedia template. If the type of the target filling material in the target multimedia template is also a content generation type, such as the target filling material itself is a picture or video generated by a generation model, etc., then the editing information of the target generation model used for the target generation material in the new multimedia template can replace the editing information of the generation model corresponding to the target filling material, such as replacing model editing information such as the model, prompt words, or model processing parameters.

[0063] As a replaceable material in the new multimedia template, in some implementation examples, the new multimedia template can retain the target generation material and add a replaceable identifier to it, which is convenient for subsequent material replacement based on this identifier. In other implementation examples, the new multimedia template deletes all replaceable materials to form a material slot to be filled, which is convenient for subsequent direct filling. Specifically, it can be flexibly set and will not be limited here. Through the above method, the embodiments of the present disclosure can quickly and efficiently generate a new multimedia template by means of the content generation function, further improving the template generation efficiency, simplifying the template generation process, and effectively expanding the template library.

[0064] Corresponding to the foregoing multimedia editing method, Figure 5 FIG. 1 is a schematic structural diagram of a multimedia editing device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is generally integrated in an electronic device, such as Figure 5 As shown, the multimedia editing device includes:

[0065] A first generation module 502, configured to obtain filling materials corresponding to a material filling operation in response to a material filling operation for a target multimedia template, so as to generate first multimedia content based on the target multimedia template and the filling materials; wherein, the target multimedia template includes editing operation information corresponding to the materials to be filled;

[0066] A function display module 504, configured to display editing functions corresponding to the target filling materials in response to receiving an editing request for the target filling materials in the filling materials; the editing functions at least include a content generation function;

[0067] A model generation module 506, configured to obtain a target generation model corresponding to the target filling materials and generate target generation materials corresponding to the target filling materials by using the target generation model when the content generation function is triggered;

[0068] A second generation module 508, configured to replace the target filling materials in the first multimedia content with the target generation materials, so as to generate second multimedia content based on the target multimedia template and the target generation materials.

[0069] The above device can provide a content generation function for the filling materials of the target multimedia template and generate corresponding materials by using a generation model, which helps to further enrich the multimedia editing effect.

[0070] In some embodiments, the model generation module 506 is specifically configured to: display selection items corresponding to multiple generation models; and obtain a target generation model corresponding to the target filling materials based on a triggered selection item among the selection items corresponding to the multiple generation models.

[0071] In some embodiments, the model generation module 506 is specifically configured to: display selection items of multiple content generation categories; wherein each content generation category corresponds to multiple generation models, and the similarity between the generation models in the same content generation category is greater than the similarity between the generation models in different content generation categories; and display selection items of multiple generation models corresponding to the target content generation category in response to a triggered selection item of the target content generation category among the selection items of the multiple content generation categories.

[0072] In some embodiments, the selection items corresponding to the generation models are presented in the form of generation effect samples corresponding to the generation models.

[0073] In some embodiments, the model generation module 506 is specifically configured to: generate a plurality of generated materials corresponding to the target filling material by using the target generation model; wherein different generated materials are generated by the target generation model using different model parameters; display the plurality of generated materials, and determine the target generated material corresponding to the target filling material based on the triggered generated material.

[0074] In some embodiments, the model generation module 506 is specifically configured to: in response to receiving an adjustment request for the triggered generated material, obtain adjustment information corresponding to the adjustment request; the adjustment information includes an updated model prompt and / or updated model processing parameters; based on the adjustment information, use the target generation model corresponding to the triggered generated material to generate the target generated material corresponding to the target filling material.

[0075] In some embodiments, the model generation module 506 is specifically configured to: display the current prompt and the current model processing parameters corresponding to the triggered generated material; wherein the triggered generated material is generated by the target generation model based on the current prompt and the current model processing parameters for processing the target filling material; provide a first modification control corresponding to the current prompt and a second modification control corresponding to the current model processing parameters; based on the operation result corresponding to the first modification control and the operation result corresponding to the second modification control, obtain the adjustment information corresponding to the adjustment request.

[0076] In some embodiments, the model generation module 506 is specifically configured to: display the target filling material and the plurality of generated materials side by side or juxtaposed, and the display position of the target filling material is before the display positions of the plurality of generated materials.

[0077] In some embodiments, the device further includes a template generation module, configured to generate a new multimedia template based on the second multimedia content; wherein the new multimedia template records the editing operation information corresponding to the replaceable target generated material, and the editing operation information of the target generated material includes the information of performing an editing operation on the target generated material by using the target generation model.

[0078] In some embodiments, the target generation model includes one or more of an image generation model, an audio generation model, and a video generation model.

[0079] The multimedia editing device provided by the embodiments of the present disclosure can execute the multimedia editing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0080] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the device embodiments described above can refer to the corresponding processes in the method embodiments and will not be elaborated herein.

[0081] Embodiments of the present disclosure provide an electronic device, which includes: a storage device storing a computer program thereon; a processing device configured to execute the computer program in the storage device to implement the steps of any one of the methods in the present disclosure.

[0082] Next, refer to Figure 6 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0083] As Figure 6 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0084] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 shows the electronic device 600 having various devices, it should be understood that it is not required to implement or include all the shown devices. More or fewer devices may be alternatively implemented or included.

[0085] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, the above-described functions defined in the methods of the embodiments of the present disclosure are performed.

[0086] In addition to the above methods and devices, embodiments of the present disclosure may also be a computer program product that includes computer program instructions that, when run by a processor, cause the processor to execute the image processing method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages for performing the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0087] Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when run by a processor, cause the processor to execute the multimedia editing method provided by the embodiments of the present disclosure.

[0088] The computer-readable storage medium may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0089] Embodiments of the present disclosure also provide a computer program product that includes a computer program / instructions that, when executed by a processor, implement the multimedia editing method in the embodiments of the present disclosure.

[0090] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained through appropriate means in accordance with relevant laws and regulations.

[0091] For example, when responding to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application program, server, or storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message.

[0092] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0093] It is understandable that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manner of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0094] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0095] The above are only specific implementation manners of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimedia editing method, characterized in that: include: In response to a material filling operation on a target multimedia template, obtaining a filling material corresponding to the material filling operation, so as to generate first multimedia content based on the target multimedia template and the filling material; wherein the target multimedia template includes editing operation information corresponding to the material to be filled; In response to receiving an editing request for a target filling material among the filling materials, displaying an editing function corresponding to the target filling material; the editing function at least includes a content generation function; When the content generation function is triggered, obtaining a target generation model corresponding to the target filling material, and using the target generation model to generate a target generation material corresponding to the target filling material; The target generation material is used to replace the target filling material in the first multimedia content to obtain the second multimedia content.

2. The method according to claim 1, characterized in that The obtaining of the target generation model corresponding to the target filling material includes: Display options corresponding to multiple generation models; Based on a triggered selection item among the selection items corresponding to the multiple generation models, a target generation model corresponding to the target filling material is obtained.

3. The method according to claim 2, characterized in that The display of the selection items corresponding to the multiple generation models includes: Displaying multiple content generation category options; wherein each content generation category corresponds to multiple generation models; and the similarity between generation models in the same content generation category is greater than the similarity between generation models in different content generation categories; In response to a selection item of a target content generation category among the selection items of the plurality of content generation categories being triggered, selection items of a plurality of generation models corresponding to the target content generation category are displayed.

4. The method according to claim 2, characterized in that: The selection items corresponding to the generation model are presented in the form of generation effect samples corresponding to the generation model.

5. The method according to claim 1, characterized in that The step of using the target generation model to generate the target generation material corresponding to the target filling material includes: Using the target generation model to generate a plurality of generation materials corresponding to the target filling material; wherein different generation materials are generated by the target generation model using different model parameters; The multiple generated materials are displayed, and a target generated material corresponding to the target filling material is determined based on the triggered generated material.

6. The method according to claim 5, characterized in that The determining, based on the triggered generation material, a target generation material corresponding to the target filling material includes: In response to receiving an adjustment request for the triggered generated material, obtaining adjustment information corresponding to the adjustment request; the adjustment information includes an updated model prompt word and / or an updated model processing parameter; Based on the adjustment information, a target generation material corresponding to the target filling material is generated using a target generation model corresponding to the triggered generation material.

7. The method according to claim 6, characterized in that The obtaining adjustment information corresponding to the adjustment request includes: Displaying the current prompt word and current model processing parameters corresponding to the triggered generation material; wherein the triggered generation material is generated by the target generation model processing the target filling material based on the current prompt word and current model processing parameters; Providing a first modification control corresponding to the current prompt word and a second modification control corresponding to the current model processing parameter; Adjustment information corresponding to the adjustment request is obtained based on the operation result corresponding to the first modification control and the operation result corresponding to the second modification control.

8. The method according to claim 5, characterized in that The display of the multiple generated materials includes: The target filling material and the multiple generated materials are displayed side by side or in parallel, and the display position of the target filling material is located before the display positions of the multiple generated materials.

9. The method according to claim 1, characterized in that: The method further comprises: A new multimedia template is generated based on the second multimedia content; wherein the new multimedia template records editing operation information corresponding to the replaceable target generation material, and the editing operation information of the target generation material includes information on editing the target generation material using the target generation model.

10. The method according to claim 1, characterized in that The target generation model includes one or more of an image generation model, an audio generation model, and a video generation model.

11. A multimedia editing device, characterized in that: include: A first generating module is configured to, in response to a material filling operation on a target multimedia template, obtain a filling material corresponding to the material filling operation, so as to generate first multimedia content based on the target multimedia template and the filling material; wherein the target multimedia template includes editing operation information corresponding to the material to be filled; A function display module, configured to display an editing function corresponding to a target filling material in the filling materials in response to receiving an editing request for the target filling material; the editing function at least includes a content generation function; A model generation module, configured to obtain a target generation model corresponding to the target filling material when the content generation function is triggered, and generate a target generation material corresponding to the target filling material using the target generation model; The second generating module is configured to replace the target filling material in the first multimedia content with the target generating material, so as to generate the second multimedia content based on the target multimedia template and the target generating material.

12. An electronic device, characterized in that: The electronic device comprises: a storage device having a computer program stored thereon; A processing device, used to execute the computer program in the storage device to implement the steps of the multimedia editing method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the multimedia editing method described in any one of claims 1 to 10.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the multimedia editing method according to any one of claims 1 to 10 when being executed by a processor.

Images