Method and apparatus for multimedia editing, device, and storage medium

Abstract

Embodiments of the present disclosure provide a method and an apparatus for multimedia editing, a device, and a storage medium. The method comprises: receiving a first message from a terminal device, the first message indicating a request to edit target multimedia; on the basis of the first message and a group of editing functions for multimedia editing, generating guidance information for a target model; and on the basis of output information of the target model, sending a second message to the terminal device, the second message being used for instructing the terminal device to perform a corresponding editing operation on the target multimedia. In this way, a user only needs to issue a multimedia editing request, and a remote device can then understand the intention of the user by calling a target model, thereby fulfilling editing requirements of the user.

Description

Method, apparatus, device and storage medium for multimedia editing

[0001] This application claims priority to the Chinese invention patent application entitled “Methods, devices, equipment and storage media for multimedia editing” filed on December 27, 2023, with application number 202311828531.5, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to methods, devices, apparatuses, and storage media for multimedia editing. Background Art

[0003] With the rapid development of information technology, various terminal devices can provide people with a variety of services in work and life. Terminal devices can provide users with editing functions to assist users in performing corresponding editing operations on multimedia data. It is expected that the user's multimedia editing needs can be met by understanding the user's intention. Summary of the Invention

[0004] In a first aspect of the present disclosure, a method for multimedia editing is provided. The method comprises: receiving a first message from a terminal device, the first message indicating a request to edit target multimedia; generating guidance information to a target model based on the first message and a set of editing functions for multimedia editing on the terminal device; and sending a second message to the terminal device based on output information of the target model, the second message being used to instruct the terminal device to perform a corresponding editing operation on the target multimedia.

[0005] In a second aspect of the present disclosure, a method for multimedia editing is provided. The method comprises: obtaining input information via an editing interface for editing target multimedia, the input information indicating an editing request for the target multimedia; sending a first message to a remote device based on the input information to generate guidance information for input into a target model; and presenting an editing result of the target multimedia in the editing interface based on a second message received from the remote device, wherein the second message is generated based on output information generated by the target model according to the guidance information.

[0006] In a third aspect of the present disclosure, a multimedia editing apparatus is provided. The apparatus includes: a receiving module configured to receive a first message from a terminal device, the first message indicating a request to edit target multimedia; a guidance information generation module configured to generate guidance information to a target model based on the first message and a set of editing functions used by the terminal device for multimedia editing; and a second message sending module configured to send a second message to the terminal device based on output information of the target model, the second message being used to instruct the terminal device to perform a corresponding editing operation on the target multimedia.

[0007] In a fourth aspect of the present disclosure, a multimedia editing apparatus is provided. The apparatus includes: an information acquisition module configured to acquire input information via an editing interface for editing target multimedia, the input information indicating an editing request for the target multimedia; a first message sending module configured to send a first message to a remote device based on the input information, for generating guidance information input to a target model; and an editing result presentation module configured to present an editing result of the target multimedia in the editing interface based on a second message received from the remote device, wherein the second message is generated based on output information generated by the target model according to the guidance information.

[0008] In a fifth aspect of the present disclosure, a remote device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the remote device to perform the method of the first aspect.

[0009] In a sixth aspect of the present disclosure, a terminal device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the terminal device to perform the method of the second aspect.

[0010] In a seventh aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect or the second aspect is implemented.

[0011] In an eighth aspect of the present disclosure, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions, which, when executed by a device, cause the device to perform the method of the first aspect or the second aspect.

[0012] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0014] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0015] 2A to 2C illustrate schematic diagrams for multimedia editing according to some embodiments;

[0016] 3A to 3C show schematic diagrams for multimedia editing according to other embodiments;

[0017] 4A and 4B show schematic diagrams for multimedia editing according to yet other embodiments;

[0018] FIG5 illustrates a flowchart of a process for multimedia editing according to some embodiments of the present disclosure;

[0019] FIG6 illustrates a flow chart of a process for multimedia editing according to some embodiments of the present disclosure;

[0020] FIG7 shows a schematic structural block diagram of an apparatus for multimedia editing according to some embodiments of the present disclosure;

[0021] FIG8 shows a schematic structural block diagram of an apparatus for multimedia editing according to some embodiments of the present disclosure; and

[0022] FIG9 illustrates a block diagram of an electronic device suitable for implementing one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0024] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0025] Herein, unless explicitly stated otherwise, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0026] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0027] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the type, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and authorization should be obtained from relevant users in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0028] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to the software or hardware such as the electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0029] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message may be sent to the relevant user in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0030] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0031] As used herein, the term "model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, corresponding outputs can be generated for given inputs. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0032] Currently, terminal devices can provide users with editing functions to assist them in performing multimedia editing operations. For example, they can edit videos, create animations, and perform other editing operations. However, traditional methods rely on users manually performing editing operations such as editing, animation, and transition selection. This method is time-consuming, difficult to manage, and the editing operations are relatively complex.

[0033] In an embodiment of the present disclosure, a multimedia editing solution is proposed. In this solution, a user's request to edit target multimedia is received from a terminal device. Based on the user's request and a set of editing functions used by the terminal device for multimedia editing, guidance information for a target model is generated. Furthermore, based on the information output by the target model, instructions for performing corresponding editing operations on the target multimedia are sent to the terminal device.

[0034] In this way, the embodiment of the present disclosure can understand the user's intention by calling the target model, and then complete the corresponding editing operation. Therefore, the embodiment of the present disclosure can complete the multimedia editing process more efficiently.

[0035] Some example embodiments of the present disclosure will be described in detail below with reference to examples in the accompanying drawings.

[0036] Sample Environment

[0037] FIG1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In this example environment 100, an application 120 is installed on a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or devices attached to the terminal device 110. In some embodiments, the application 120 may include a multimedia editing application or any suitable application with integrated multimedia editing capabilities.

[0038] Application 120 can provide the user 140 with an integration of multiple applications or components. These applications can be used as application modules in application 120. In some embodiments, application 120 can be downloaded and installed as an application on terminal device 110. In some embodiments, application 120 can also be accessed through other means, such as through a web page.

[0039] In the environment 100 of FIG1 , if the application 120 is started, the terminal device 110 may present an interface 150 of the application 120 to the user 140. The interface 150 is sometimes also referred to as a client interface. The interface 150 may include various interfaces provided by the application 120, such as a conversation interface for presenting chat content, an editing interface, an editing result interface, and so on.

[0040] In some embodiments, terminal device 110 communicates with remote device 130 to implement the provision of services for application 120. A target model 160 is deployed in environment 100. Target model 160 can be deployed on an appropriate electronic device. For example, target model 160 can also be deployed on an electronic device other than remote device 130. Remote device 130 can, for example, invoke target model 160 through a service call to perform a corresponding task. In another example, target model 160 can also be deployed locally on remote device 130.

[0041] As will be described in detail below, terminal device 110 can use target model 160 to understand the user's editing intent and automatically complete the corresponding multimedia editing operation. As an example, terminal device 110 can call the target model through remote device 130; as another example, terminal device 110 can directly call the target model to understand the user's editing intent.

[0042] The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a multimedia computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.). The remote device 130 can be various types of computing systems / servers that can provide computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0043] It should be understood that the structure and function of each element in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. For example, the embodiments of the present disclosure can be applied to any suitable one or more applications, and are not limited to multimedia editing.

[0044] Example Process

[0045] It should be understood that the examples described in the present disclosure can be implemented by the terminal device 110, the remote device 120, or a combination of the terminal device 110 and the remote device 120 as shown in FIG1 . Therefore, although in some embodiments, corresponding functions are described as being implemented by the terminal device 110, it should be understood that in other embodiments, corresponding functions may also be implemented by the remote device 130. Similarly, although in some embodiments, corresponding functions are described as being implemented by the remote device 130, it should be understood that in other embodiments, corresponding functions may also be implemented by the terminal device 110. The disclosed embodiments are not limited in terms of the execution subject.

[0046] For the sole purpose of better understanding the various embodiments of the present disclosure, reference is made to the example user interfaces 201 to 402 shown in Figures 2A to 4B in the example embodiments below. It should be understood that the user interfaces shown in Figures 2A to 4B are merely examples, and various designs may exist. For example, the various graphical elements and / or controls in the user interface may have different arrangements and different visual representations, one or more elements and / or controls therein may be omitted or replaced, and one or more other elements and / or controls may also exist. In addition, any appropriate content may be included in the user interface. The scope of the present disclosure is not limited in this respect.

[0047] First, the solution for multimedia editing disclosed herein will be described with reference to Figures 2A to 2C. Figures 2A to 2C show schematic diagrams for multimedia editing according to some embodiments. For ease of description, Figures 2A to 2C will be described with reference to Figure 1.

[0048] Terminal device 110 obtains input information 220 from user 140 via interface 150. Interface 150 is an editing interface for editing target multimedia 210. Input information 220 from user 140 represents user 140's editing requirements for target multimedia 210, i.e., a request to edit the target multimedia. In some examples, terminal device 110 may obtain input information 220 through text content, voice content, etc. input by user 140. Terminal device 110 may also receive a triggering operation from user 140 on a control of at least one editing item in interface 150 by user 140 clicking on the control of at least one editing item.

[0049] In some embodiments, terminal device 110 sends a first message to remote device 130 based on input information 220 of user 140. Remote device 130 generates guidance information for input to target model 160 based on the first message. The generation of guidance information for target model 160 by remote device 130 will be described in detail below.

[0050] Based on the second message received from remote device 130, terminal device 110 presents the edit results of target multimedia 210 on editing interface 150. As shown in Figure 3C, terminal device 110 displays edit results 310 on the editing page and responds with "Cutting completed, ready for adjustment." Figure 3C is described in detail below. In some examples, the second message is generated by terminal device 110 based on output information generated by target model 160 using guidance information.

[0051] In some examples, the editing interface 150 includes an interactive area, in which the terminal device 110 can provide feedback indicating that at least one editing operation has been applied to the target multimedia. In some examples, the terminal device 110 can associate the feedback message with an adjustment control. As shown in FIG3C , the terminal device 110 provides an adjustment control 311, and the user 140 can click on the adjustment control 311 to adjust the editing result 310.

[0052] In some embodiments, the terminal device 110 modifies a parameter based on the user 140 clicking the adjustment control 311. The parameter is a parameter associated with at least one editing operation of the target multimedia.

[0053] As shown in Figures 2A to 2C , terminal device 110 receives input from user 140 in input box 214, such as "Help me add a sentence here," through editing interface 150. Terminal device 110 sends a first message to remote device 130 based on user 140's input of "Help me add a sentence here." Remote device 130 generates guidance information based on the first message, which is then input into target model 160.

[0054] From the moment terminal device 110 receives input from user 140 until it receives the second message, terminal device 110 displays a processing progress 320 on editing interface 150, such as "Recommending content, please wait." Based on the displayed progress, user 140 can also click a cancel control to pause the editing operation. Based on the second message received from remote device 130, terminal device 110 presents the edited result 310 of target multimedia 210 on editing interface 150.

[0055] The following describes a multimedia editing solution with examples and in conjunction with Figures 3A to 3C and Figures 4A to 4B. Figures 3A to 3C show schematic diagrams for multimedia editing according to some other embodiments.

[0056] As shown in Figures 3A to 3C , terminal device 110 receives input from user 140 in input box 214, such as "Help me cut out," through editing interface 150. Terminal device 110 sends a first message to remote device 130 based on user 140's input of "Help me cut out." Remote device 130 generates guidance information based on the first message, which is then input into target model 160.

[0057] From the moment terminal device 110 receives input from user 140 until it receives the second message, terminal device 110 displays a processing progress 320 on editing interface 150, such as "Cutting out your image, please wait 64%." Based on the displayed progress, user 140 can also click a cancel control to pause the editing operation. Based on the second message received from remote device 130, terminal device 110 presents the edited result 310 of target multimedia 210 on editing interface 150.

[0058] Figures 4A and 4B illustrate schematic diagrams for multimedia editing according to further embodiments. As shown in Figures 4A and 4B, terminal device 110 receives input from user 140 in input box 214, such as "adjust beauty face," via editing interface 150. Terminal device 110 sends a first message to remote device 130 based on user 140's input of "adjust beauty face." Remote device 130 generates guidance information based on the first message, which is then input into target model 160.

[0059] Terminal device 110 presents editing result 410 of target multimedia 210 in editing interface 150 according to the second message received from remote device 130. User 140 adjusts editing result 410 based on adjustment controls presented in the form of frame 420 provided by terminal device 110 in the interactive area.

[0060] The above description of the multimedia editing solution is from the perspective of terminal device 110. Terminal device 110 receives user input and sends a first message to remote device 130. Furthermore, terminal device 110 receives a second message from remote device 130 and presents the editing results. The following description, from the perspective of remote device 130, describes how guidance information for generating target model 160 based on the first message and a set of APIs for multimedia editing is generated.

[0061] In some embodiments, remote device 130 receives a first message from terminal device 110. The first message indicates a request to edit target multimedia. The first message may be content input by user 140 into terminal device 110 on interface 150. The input content may be textual input, voice input, or the like. As shown in FIG2B , user 140 enters "Help me add a sentence here" into input box 214 of interface 150. In some examples, multimedia includes videos, images, and composite multimedia comprising multiple multimedia types.

[0062] In some examples, if the input content is of text type, the guidance information of the target model 160 includes a text related to the input text to represent the content input by the user 140. If the input content is of voice type, the target model 160 transcribes the input voice into text.

[0063] The first message can also be a triggering operation of the user 140 on at least one editing item provided by the terminal device 110 on the interface 150. As shown in Figure 2A, the editing items provided by the terminal device 110 include: changing music 211, adding packaging 212, adding copy 213, etc. The user 140 can click on any editing item provided by the terminal device 110. In some examples, if the user 140 chooses to click on any editing item, the guidance information of the target model 160 needs to include a paragraph of text corresponding to any editing item. The paragraph of text corresponding to any editing item included in the guidance information of the target model 160 is used to indicate that the user 140 has clicked on any editing item.

[0064] In some embodiments, the remote device 130 generates guidance information (also called guide words or prompt words) of the target model 160 according to the first message and a set of editing functions used by the terminal device for multimedia editing.

[0065] In some embodiments, the guide information may include one or more of the following information items: capability setting information, material information, a set of editing functions, at least one example, a state of the material, and user input.

[0066] In some embodiments, the set of editing functions may be a set of application programming interfaces (APIs) for multimedia editing. In some examples, the set of APIs may be an API for replacing all flower characters in a text in multimedia, an API for adding a certain music to multimedia, and so on, which is not limited by the present disclosure.

[0067] Additionally, in some embodiments, a set of editing functions may include a description of the editing function, an explanation, an editing function declaration, a processing example, a user status description, and a user interaction option description. The editing function description may include a "background" parameter, a "type" parameter, an "identifier" parameter, a "material" parameter, and so on. The editing function explanation may include parameters such as video, audio, music, text_to_audio, text, lyrics, and subtitle. The editing function declaration may include parameters associated with "speed change." For example, slowing down to 0.8x speed can be expressed as [0.8], speeding up 50% can be expressed as [1.5x], and so on. If the user's status is ['video_7'] material, use the Python function to answer the questions described in the user interaction option description: replace all scrambled characters, output code, and package it in "Python" format. If the question is irrelevant to editing or difficult to complete, return the "unsupported()" function and do not force an answer.

[0068] For example, a set of editing functions might include the following: When using the add-related functions (xxAdd), you need to create new draft_ids as the function input; otherwise, use existing draft_ids as input. When using the edit-related functions (xxEdit), if the question does not specify which material to operate on, perform the operation on the first material in the material. Dubbing uses synthesized voices; recording is not required. For copyright verification and replacement, or commercial use of materials, use the materialEdit function, such as changing copyrighted music or fonts.

[0069] In some embodiments, the remote device 130 generates guidance information for the target model 160 based on the first message and a set of editing functions received from the terminal device 110. The guidance information may be a set of materials associated with the target multimedia to be edited. For example, if the target multimedia to be edited is a video, the set of materials associated with it may include audio, text, stickers, filters, special effects, and so on.

[0070] For example, a set of editing functions for performing video editing operations may include, but are not limited to, replacement, layout modification, subtitle recognition, time stamping, speed change, shot tracking, cropping, scaling, background change, background editing, packaging and polishing. The output information of target model 160 may include parameters associated with "replacement." The output information of target model 160 may also include parameters associated with "layout modification," such as parameters associated with rotating, moving, and resizing a video. The output information of target model 160 may also include parameters associated with "speed change," such as slowing down to 0.8 times the speed using [0.8], speeding up by 50% using [1.5x], and so on.

[0071] In some embodiments, the guidance information also includes a capability setting section, for example, "You are a professional video editing program from XYZ. Your job is to use Python functions to modify the API based on user questions, minimizing the effect the user wants. You only output code."

[0072] The guidance information also indicates the processing status of at least one material. For example, the guidance information needs to reflect the user's selection status of the material and inform the user that a certain material in the video is currently selected. After the guidance information reflects the material currently selected by the user, if the user has other editing requests, the guidance information also needs to reflect the answer based on the user's editing request. For example, the target model 160 is informed: "If the user's current selection status is to select the ['video_7'] material, please use the python function to answer the user's question: replace all flower characters."

[0073] The guidance information also indicates at least one processing example. The at least one processing example includes an editing request and a corresponding model output. For example, the target model 160 is informed that when the user's multimedia editing requirement is to bold the subtitles, the output can be textEdit('replace',['all_subtitles'],'{ori_str}','re.sub(".+",\' \'+str+\' \',str)'); when the user's multimedia editing requirement is to translate the subtitles into French, the output can be textEdit("translation",['all_subtitles'],'fr'); when the user's multimedia editing requirement is to delete the bottom subtitles, the output can be materialDelete(['all_subtitles']), and so on.

[0074] In some embodiments, remote device 130 sends a second message to terminal device 110 based on the output information of target model 160. The second message instructs terminal device 110 to perform a corresponding editing operation on the target multimedia. In some examples, the output information of target model 160 may be a parameter associated with at least one editing function from a set of editing functions.

[0075] In some embodiments, remote device 130 may generate a second message based on the output information of target model 160. The second message is code executable by terminal device 110, for example, code in TypeScript format. Remote device 130 sends the second message to terminal device 110, which then presents the edited result of the target multimedia in editing interface 150. In some examples, the output information of target model 160 includes a first code representation corresponding to a first programming language. The second message includes a second code representation corresponding to a second programming language.

[0076] The disclosed embodiment receives a user's request to edit target multimedia from a terminal device and generates guidance information for a target model based on the user's request and a set of editing functions for multimedia editing. Based on the information output by the target model, the remote device sends instructions to the terminal device for performing the corresponding editing operations on the target multimedia. In this way, the user only needs to issue a request to edit the multimedia, and the remote device can understand the user's intention by calling the target model and then implement the user's editing request.

[0077] Example Method

[0078] FIG5 shows a flowchart of a process 500 for multimedia editing according to some embodiments of the present disclosure. Process 500 can be implemented by remote device 130, terminal device 110, or a combination of terminal device 110 and remote device 130 in FIG1 . For ease of description, process 500 is described as being implemented at remote device 130.

[0079] In block 510 , the remote device 130 receives a first message from a terminal device, the first message indicating a request to edit target multimedia.

[0080] In block 520 , the remote device 130 generates guidance information to the target model based on the first message and a set of editing functions of the terminal device for multimedia editing.

[0081] In block 530 , the remote device 130 sends a second message to the terminal device based on the output information of the target model, where the second message is used to instruct the terminal device to perform a corresponding editing operation on the target multimedia.

[0082] In some embodiments, the first message is determined based on at least one of the following: content input by a user to the terminal device; or a triggering operation of at least one editing item provided by the user to the terminal device.

[0083] In some embodiments, the guidance information indicates: a set of materials associated with the target multimedia to be edited; a processing status of at least one material; or at least one processing example, the at least one processing example including an editing request and a corresponding model output.

[0084] In some embodiments, the output information indicates: at least one target edit function from a set of edit functions; and parameters associated with the at least one target edit function.

[0085] In some embodiments, process 500 further includes: generating a second message based on the output information, the second message including a code representation executed by the terminal device.

[0086] In some embodiments, the output message includes a first code representation corresponding to a first programming language, and the second message includes a second code representation corresponding to a second programming language.

[0087] FIG6 illustrates a flowchart of a process 600 for multimedia editing according to some embodiments of the present disclosure. Process 600 may be implemented at the terminal device 110 of FIG1 , or implemented at or included in other devices for multimedia editing, such as other terminal devices or service devices. For ease of description, it is assumed that process 600 may be implemented at the terminal device 110, and process 600 will be described below with reference to FIG1 .

[0088] In block 610 , the terminal device 110 obtains input information via an editing interface for editing target multimedia, where the input information indicates an editing request for the target multimedia.

[0089] In block 620 , the terminal device 110 sends a first message to the remote device based on the input information, so as to generate guidance information to be input into the target model.

[0090] In block 630 , the terminal device 110 presents the editing result of the target multimedia in the editing interface based on the second message received from the remote device, wherein the second message is generated based on the output information generated by the target model according to the guidance information.

[0091] In some embodiments, obtaining input information includes: obtaining text content and / or voice content input by a user; and receiving a trigger operation of the user on at least one editing item in the editing interface.

[0092] In some embodiments, the editing interface includes an interactive area, and the method further includes: providing a feedback message in the interactive area, the feedback message indicating that at least one editing operation has been applied to the target multimedia.

[0093] In some embodiments, process 600 further includes: providing an adjustment control in association with the feedback message; and modifying an editing parameter associated with at least one editing operation applied to the target multimedia based on a selection of the adjustment control.

[0094] Example devices and equipment

[0095] Embodiments of the present disclosure also provide corresponding devices for implementing the above-mentioned methods or processes. FIG7 shows a schematic structural block diagram of a device 700 for multimedia editing according to some embodiments of the present disclosure. The device 700 can be implemented or included in the remote device 130 and / or the terminal device 110, or implemented or included in other devices for multimedia editing, such as other terminal devices or service devices. The various modules / components in the device 700 can be implemented by hardware, software, firmware, or any combination thereof.

[0096] As shown in the figure, the apparatus 700 includes a receiving module 710 configured to receive a first message from a terminal device, where the first message indicates a request to edit target multimedia.

[0097] The apparatus 700 further includes a guidance information generating module 720 configured to generate guidance information to the target model based on the first message and a set of editing functions used by the terminal device for multimedia editing.

[0098] The apparatus 700 further includes a second information sending module 730 configured to send a second message to the terminal device based on the output information of the target model, where the second message is used to instruct the terminal device to perform a corresponding editing operation on the target multimedia.

[0099] In some embodiments, the receiving module 710 further includes a first message determining module configured to be content input by a user to the terminal device; or a triggering operation of at least one editing item provided by the user to the terminal device.

[0100] In some embodiments, the guidance information further indicates: a set of materials associated with the target multimedia to be edited; a processing status of at least one material; or at least one processing example, wherein the at least one processing example includes an editing request and a corresponding model output.

[0101] In some embodiments, the output information indicates: at least one target edit function from a set of edit functions; and parameters associated with the at least one target edit function.

[0102] In some embodiments, the second information sending module 730 further includes a second message generating module configured to generate a second message based on the output information, where the second message includes a code representation executed by the terminal device.

[0103] In some embodiments, the output message includes a first code representation corresponding to a first programming language, and the second message includes a second code representation corresponding to a second programming language.

[0104] FIG8 shows a schematic block diagram of an apparatus 800 for multimedia editing according to some embodiments of the present disclosure. Apparatus 800 may be implemented or included in, for example, terminal device 110, or implemented or included in other devices for multimedia editing, such as other terminal devices or service devices. Each module / component in apparatus 800 may be implemented by hardware, software, firmware, or any combination thereof.

[0105] As shown in the figure, the apparatus 800 includes an information acquisition module 810 configured to acquire input information via an editing interface for editing target multimedia, where the input information indicates an editing request for the target multimedia.

[0106] The apparatus 800 further includes a first message sending module 820 configured to send a first message to a remote device based on the input information, so as to generate guidance information input to the target model.

[0107] The apparatus 800 further includes an editing result presenting module 830 configured to present the editing result of the target multimedia in the editing interface based on a second message received from a remote device, wherein the second message is generated based on output information generated by the target model according to the guidance information.

[0108] In some embodiments, the information acquisition module 810 is further configured to acquire text content and / or voice content input by the user; and receive a trigger operation of the user on at least one editing item in the editing interface.

[0109] In some embodiments, the editing interface includes an interactive area, the editing result presentation module 830 is further configured to provide a feedback message in the interactive area, the feedback message indicating that at least one editing operation has been applied to the target multimedia.

[0110] In some embodiments, the editing result presentation module 830 is further configured to provide an adjustment control in association with the feedback message; and modify an editing parameter associated with at least one editing operation applied to the target multimedia based on a selection of the adjustment control.

[0111] As shown in FIG9 , electronic device 900 is a general-purpose electronic device. Components of electronic device 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. Processing unit 910 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 920. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 900.

[0112] The electronic device 900 typically includes a plurality of computer storage media. Such media can be any accessible media that the electronic device 900 can access, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 920 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 930 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 900.

[0113] The electronic device 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 9 , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 920 may include a computer program product 925 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0114] The communication unit 940 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 900 can be implemented as a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 900 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0115] The input device 950 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 960 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 900 may also communicate with one or more external devices (not shown) through the communication unit 940 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 900, or with any device that allows the electronic device 900 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0116] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0117] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0118] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0119] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0120] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0121] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A multimedia editing method, comprising: Receiving a first message from a terminal device, the first message indicating a request to edit a target multimedia; Generating guiding information to a target model based on the first message and a set of editing functions used by the terminal device for multimedia editing; And Sending a second message to the terminal device based on the output information of the target model, the second message being used to instruct the terminal device to perform a corresponding editing operation on the target multimedia.

2. The method according to claim 1, wherein the first message is determined based on at least one of the following: Content input by a user to the terminal device; or A triggering operation of the user on at least one editing item provided by the terminal device.

3. The method according to claim 1, wherein the guiding information indicates: A set of materials associated with the target multimedia to be edited; The processing status of at least one material; or At least one processing example, the at least one processing example including an editing request and a corresponding model output.

4. The method according to claim 1, wherein the output information indicates: At least one target editing function in the set of editing functions; and Parameters associated with the at least one target editing function.

5. The method according to claim 1, further comprising: Generating the second message based on the output information, the second message including a code representation to be executed by the terminal device.

6. The method according to claim 5, wherein the output information includes a first code representation corresponding to a first programming language, and the second message includes a second code representation corresponding to a second programming language.

7. A multimedia editing method, comprising: Obtaining input information via an editing interface for editing a target multimedia, the input information indicating an editing request for the target multimedia; Sending a first message to a remote device based on the input information for generating guiding information input to a target model; And Presenting an editing result of the target multimedia in the editing interface based on a second message received from the remote device, wherein the second message is generated based on output information generated by the target model according to the guiding information.

8. The method according to claim 7, wherein obtaining the input information includes: Obtaining text content and / or voice content input by a user; Receiving a triggering operation of the user on at least one editing item in the editing interface.

9. The method according to claim 7, wherein the editing interface includes an interaction area, and the method further comprises: Providing a feedback message in the interaction area, the feedback message indicating that at least one editing operation has been applied to the target multimedia.

10. The method according to claim 9, further comprising: Providing an adjustment control associated with the feedback message; And Modifying editing parameters associated with the at least one editing operation applied to the target multimedia based on a selection of the adjustment control.

11. An apparatus for multimedia editing, comprising: A receiving module, configured to receive a first message from a terminal device, where the first message indicates a request to edit a target multimedia; A guiding information generating module, configured to generate guiding information to a target model based on the first message and a set of editing functions used by the terminal device for multimedia editing; And A second message sending module, configured to send a second message to the terminal device based on output information of the target model, where the second message is used to instruct the terminal device to perform a corresponding editing operation on the target multimedia.

12. An apparatus for multimedia editing, comprising: An information acquisition module, configured to acquire input information via an editing interface for editing a target multimedia, where the input information indicates an editing request for the target multimedia; A first message sending module, configured to send a first message to a remote device based on the input information for generating guiding information input to a target model; And A presenting editing result module, configured to present an editing result of the target multimedia in the editing interface based on a second message received from the remote device, where the second message is generated based on output information generated by the target model according to the guiding information.

13. A remote device, comprising: At least one processing unit; And At least one memory, where the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the remote device executes the method according to any one of claims 1 to 6.

14. A terminal device, comprising: At least one processing unit; And At least one memory, where the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the terminal device executes the method according to any one of claims 7 to 10.

15. A computer-readable storage medium, on which a computer program is stored, and the computer program can be executed by a processor to implement the method according to any one of claims 1 to 6 or claims 7 to 10.

16. A computer program product, where the computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions, and when the computer-executable instructions are executed by a device, the device executes the method according to any one of claims 1 to 6 or claims 7 to 10.

Images