Media content generation method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides a media content generation method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring input information; a task list is determined based on the input information through the first processing model, the task list comprises task information of multiple tasks, and the task information comprises identification information and task description information of the to-be-called object; determining a target to-be-called object based on the task list, determining an object parameter corresponding to the task description information through a second processing model associated with the target to-be-called object, and determining output content corresponding to the task based on the object parameter through the target to-be-called object; and generating the target media content based on the output content of the target to-be-called object. According to the embodiment of the invention, task arrangement and object calling are rapidly completed in combination with the first processing model and the second processing model, the time consumption of a media content generation process is shortened, and more model capabilities can be accessed, so that accurate generation and rapid presentation of a media content effect are realized.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly, to a media content generation method, device, equipment, medium and program product. BACKGROUND

[0002] With the development of computer technology and network technology, the functions of multimedia editing tools are becoming more and more diversified. For example, combining artificial intelligence with multimedia editing tools can enable multimedia editing tools to understand user needs and generate media content. However, the architecture of the multimedia editing tools combined with artificial intelligence in the related art is difficult to expand the number of objects to be called, the media content generation takes a long time, and the editing and generation effect of the multimedia content is poor, resulting in poor user experience. SUMMARY

[0003] Embodiments of the present disclosure provide a media content generation method, device, equipment, medium and program product, which can optimize the architecture of the multimedia editing tool, shorten the media content generation time, and improve the user experience.

[0004] In a first aspect, the embodiments of the present disclosure provide a media content generation method, comprising:

[0005] obtaining input information, wherein the input information comprises text;

[0006] determining a task list based on the input information by a first processing model, wherein the task list comprises task information of a plurality of tasks, and the task information comprises identification information of an object to be called and task description information;

[0007] determining a target object to be called based on the task list, determining an object parameter corresponding to the task description information by a second processing model associated with the target object to be called, and determining output content corresponding to the task based on the object parameter by the target object to be called;

[0008] generating target media content based on the output content of the target object to be called.

[0009] In a second aspect, the embodiments of the present disclosure also provide a media content generation device, comprising:

[0010] an information obtaining module configured to obtain input information, wherein the input information comprises text;

[0011] a task determining module configured to determine a task list based on the input information by a first processing model, wherein the task list comprises task information of a plurality of tasks, and the task information comprises identification information of an object to be called and task description information;

[0012] The object invocation module is used to determine the target invocation object based on the task list, determine the object parameters corresponding to the task description information through the second processing model associated with the target invocation object, and determine the output content corresponding to the task based on the object parameters through the target invocation object.

[0013] The content generation module is used to generate target media content based on the output content of the target object to be invoked.

[0014] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0015] One or more processors;

[0016] Storage device for storing one or more programs.

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the media content generation method as described in any embodiment of this disclosure.

[0018] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the media content generation method as described in any embodiment of this disclosure.

[0019] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the media content generation method as described in any embodiment of this disclosure.

[0020] The embodiment of the present disclosure provides a media content generation scheme, a task list is determined based on input information through a first processing model, the task list includes task information of multiple tasks, the task information includes identification information of a to-be-called object and task description information, the first processing model understands the input information and determines multiple tasks required to achieve an expected effect corresponding to the input information, the first processing model does not need to be called multiple times, and consumption of the first processing model is reduced. A target to-be-called object is determined based on the task list, an object parameter corresponding to the task description information is determined through a second processing model associated with the target to-be-called object, the second processing model associated with the to-be-called object is used to translate the task description information into the object parameter, the object parameter does not need to be determined through the first processing model, so that the context information of the first processing model is reduced, and the number of to-be-called objects is facilitated to be expanded. Output content of the task is determined based on the object parameter through the target to-be-called object, and target media content is generated based on the output content of the target to-be-called object. The embodiment of the present disclosure combines the first processing model and the second processing model to quickly complete task arrangement and object calling, shortens the time consumption of the media content generation process, and can access more model capabilities, so as to realize accurate generation and rapid presentation of media content effects. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0022] Figure 1 A flowchart of a media content generation method provided by an embodiment of the present disclosure;

[0023] Figure 2 A flowchart of another media content generation method provided by an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of still another media content generation method provided by an embodiment of the present disclosure;

[0025] Figure 4 A structural diagram of a media content generation device provided by an embodiment of the present disclosure;

[0026] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein; rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related terms are defined in the following description.

[0030] It should be noted that the terms "first", "second", and the like in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the adjectives "one", "more" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that "one or more" should be understood unless the context clearly indicates otherwise.

[0032] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of the messages or information.

[0033] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0034] For example, when responding to the active request of the user, the user is sent a prompt message to explicitly prompt the user that the operation requested to be performed will require the acquisition and use of the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware such as electronic devices, application programs, servers or storage media that perform the operation of the technical solutions of the present disclosure according to the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving the active request of the user, the manner of sending the prompt information to the user may be, for example, a pop-up window manner, in which the prompt information may be presented in the form of text. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0036] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation of the present disclosure. Other manners that meet relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0037] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0038] Figure 1 A flowchart of a media content generation method provided by the embodiments of the present disclosure is shown. The embodiments of the present disclosure are applicable to the case of multimedia creation, for example, the scenario of generating a picture and / or a video based on user input text. Alternatively, the scenario of generating a picture and / or a video based on user input text and a picture, etc. The method can be executed by a media content generation device, which can be implemented in the form of software and / or hardware. Alternatively, it can be implemented by an electronic device, which can be a mobile terminal, a PC terminal, or a server, etc.

[0039] As shown in Figure 1 , the method comprises:

[0040] S110, obtaining input information, wherein the input information comprises text.

[0041] Exemplarily, the input information can be text input by the user, etc. The user can input text through an input box in an interactive page. Alternatively, the input information further comprises at least one of voice, picture, and video. In some embodiments, the user can also input voice information through a voice input control in the interactive page, and obtain the input text through voice-to-text processing. Alternatively, the text converted from the voice information can be displayed in the input box. Alternatively, the user can also input a picture or a video as reference content for generation operation or editing operation.

[0042] S120, determining a task list based on the input information through a first processing model, wherein the task list comprises task information of a plurality of tasks, and the task information comprises identification information of a to-be-invoked object and task description information.

[0043] The first processing model is configured to output a plurality of tasks for implementing an intent corresponding to the input information based on the input information. In some embodiments, the first processing model understands the intent corresponding to the input information, and determines a to-be-invoked object to be invoked and a task to be performed by the to-be-invoked object for implementing the intent corresponding to the input information. The plurality of tasks are presented in the form of a task list. For example, the first processing model can be a deep learning model for processing natural language. Alternatively, the first processing model can be a deep learning model for processing multi-modal data, etc. The task list represents the presentation form of the task information of the plurality of tasks. The tasks in the task list are associated with steps for implementing the intent corresponding to the input information. By decomposing the intent corresponding to the input information into a plurality of steps through the first processing model, a complex intent can be split into smaller and simpler steps, and the intent can be ultimately implemented by implementing each small step. In the embodiments of the present disclosure, each small step corresponds to a task described by natural language.

[0044] The task information includes identification information of the to-be-invoked object and task description information. The to-be-invoked object represents the execution subject of the task. The to-be-invoked object indicates who performs the task. The identification information of the to-be-invoked object can include a to-be-invoked object name or a to-be-invoked object number, etc. The task description information is a task described by natural language. The task description information includes what the to-be-invoked object needs to do. In some embodiments, the task information further includes input content information and output content information of the task. The input content information of the task represents content information on which the task execution depends. The output content information of the task represents content information generated after the task execution is completed. For example, the input content information can include the name of a picture and / or a video on which the task execution depends. The picture and / or the video are obtained from a preset storage space based on the name. The output content information can include the name of a picture or a video generated after the task execution is completed. The picture or the video generated after the task execution is completed is stored in the preset storage space.

[0045] For example, the determining, by the first processing model, the task list based on the input information includes: inputting the input information into the first processing model, determining, by the first processing model, an intent attribute of the input information, generating a plurality of tasks based on the intent attribute, and outputting a task list corresponding to the tasks. In the embodiments of the present disclosure, the first processing model is called once for input information to understand the intent attribute of the input information, and the intent attribute is split into a plurality of tasks, without the need for multiple invocations of the first processing model to implement task arrangement.

[0046] In some embodiments, the system prompt word of the first processing model includes role description and task requirements, etc. The system prompt word prompts the first processing model to understand the intent attribute of the input information, and to split the intent of the input information into a plurality of tasks described by natural language to obtain a task list.

[0047] In S130, a target to-be-invoked object is determined based on the task list, an object parameter corresponding to the task description information is determined through a second processing model associated with the target to-be-invoked object, and output content corresponding to the task is determined through the target to-be-invoked object based on the object parameter.

[0048] The target to-be-invoked object can be an object that executes a task in the list. For example, the target to-be-invoked object can include one or more of a picture generation tool, a picture editing tool, a text generation tool, a video generation tool, and a video editing tool. For example, the picture generation tool can include a tool that implements a text-to-picture function. The picture editing tool can include a tool that implements a picture-to-picture function. The text generation tool can include a tool that generates text between pictures and / or videos. The video generation tool can include a tool that implements a text-to-video function. The video editing tool can include a tool that implements a video-to-video function.

[0049] The second processing model refers to a natural language processing model associated with the to-be-invoked object. The second processing model is used to translate the task description information into an object parameter of the to-be-invoked object. For example, the object parameter includes a parameter required for the to-be-invoked object to execute the task. For example, the object parameter includes a prompt word, a picture resolution, a style, and the like. In some embodiments, the second processing model associated with the text-to-picture tool generates a prompt word of the text-to-picture tool based on the task description information, to prompt the text-to-picture tool to execute a picture generation operation. The second processing model associated with the picture-to-picture tool generates a prompt word of the picture-to-picture tool based on the task description information, to prompt the picture-to-picture tool to execute a picture editing operation based on the prompt word and a picture represented by the input content information. The functions of the video tool are similar to those of the picture tool, and are not described here. The second processing model associated with the text generation tool generates a prompt word of the text-to-picture tool based on the task description information, to prompt the text generation tool to execute a text generation operation.

[0050] The embodiments of the present disclosure can avoid defining constraint information of each to-be-invoked service in the system prompt word of the first processing model by deploying the second processing model on the node corresponding to the to-be-invoked service, translating the task description information into a service parameter through the second processing model, and inputting the service parameter into the to-be-invoked service. This greatly saves the context in the system prompt word of the first processing model. If the constraint information of each to-be-invoked service is defined in the system prompt word of the first processing model, it is difficult to expand the to-be-invoked object, because each increase of a to-be-invoked object requires a large amount of constraint information about the model boundary to be added to the system prompt word, which will cause the framework to become extremely complex. The embodiments of the present disclosure only need to add the corresponding second processing model if a new to-be-invoked service is added, and the number of to-be-invoked objects cannot be expanded due to the limitation of the system prompt word of the first processing model.

[0051] Exemplarily, the task execution sequence is determined based on the task list, and the target task is determined based on the task execution sequence. The identification information of the target callable object in the task information of the target task is determined based on the identification information of the callable object. The object parameter corresponding to the task description information is determined through the second processing model associated with the target callable object, and the output content corresponding to the task is determined by the target callable object based on the object parameter.

[0052] In some embodiments, determining the object parameter corresponding to the task description information through the second processing model associated with the target callable object comprises: sending the task description information to the second processing model associated with the target callable object. The task description information is converted into the object parameter by the second processing model based on preset prompt information, and the object parameter is output to the callable object, wherein the preset prompt information comprises constraint information for generating the object parameter of the callable object.

[0053] The object parameter represents the input parameter required by the callable object to execute the task. The preset prompt information comprises requirement information of the prompt word corresponding to the callable object. For example, the preset prompt information comprises at least one of construction requirement, resolution requirement, and style requirement of the prompt word.

[0054] Exemplarily, the task description information is sent to the second processing model associated with the target callable object, and the object parameter of the target callable object output by the second processing model is obtained. The object parameter is input into the target callable object, so that the target callable object executes the task based on the object parameter to obtain the output content of the target callable object. In the embodiment of the disclosure, the natural language form of the task description information is translated into the object parameter of the target callable object through the second processing model, the task description information of the target callable object is translated through the second processing model adapted to the target callable object, the object parameter of the target callable object is obtained, and the first processing model does not need to output the object parameter to the target callable object. Therefore, the constraint information of each callable object does not need to be included in the system prompt word of the first processing model, and the context of the first processing model is reduced.

[0055] For example, if the target callable object is a text-to-image tool, the task description information is sent to the second processing model associated with the text-to-image tool, so that the tool parameter of the text-to-image tool is determined based on the task description information under the constraint of the preset prompt information through the second processing model associated with the text-to-image tool. The output content corresponding to the current task is determined by the text-to-image tool based on the tool parameter.

[0056] Alternatively, if the target to-be-invoked object is a graph generation tool, the task description information is sent to a second processing model associated with the graph generation tool, so that the tool parameters of the graph generation tool are determined based on the task description information under the constraint of preset prompt information by the second processing model associated with the graph generation tool. The output content corresponding to the current task is determined by the graph generation tool based on the tool parameters.

[0057] Alternatively, if the target to-be-invoked object is a text generation tool, the task description information is sent to a second processing model associated with the text generation tool, so that the tool parameters of the text generation tool are determined based on the task description information under the constraint of preset prompt information by the second processing model associated with the text generation tool. The output content corresponding to the current task is determined by the text generation tool based on the tool parameters.

[0058] S140, generating target media content based on the output content of the target to-be-invoked object.

[0059] The target media content is a generation result corresponding to the intent attribute of the input information. The target media content includes pictures and / or videos.

[0060] Illustratively, the output text and media material returned by the to-be-invoked object are obtained, wherein the media material includes output pictures and / or output videos. The output text and media material are arranged based on the order of the tasks in the task list, to obtain the target media content.

[0061] In some embodiments, the target media content includes single-round generation of an output picture or an output video, and the output text is displayed above the output picture or the output video, and the summary content of the content of the output picture or the output video is displayed through the output text. The summary content includes at least one of picture description information, resolution, style, etc.

[0062] Alternatively, the target media content includes single-round generation of at least two output pictures or output videos, and the output text is displayed above the output pictures and / or the output videos, and the output text is displayed between adjacent output pictures or adjacent output videos, and the output text is displayed between adjacent output pictures and output videos, and the summary content is displayed through the output text.

[0063] The technical scheme of the embodiments of the present disclosure determines a task list based on input information through a first processing model, the task list includes task information of multiple tasks, the task information includes identification information of a to-be-invoked object and task description information, the first processing model understands the input information and determines multiple tasks required to achieve the expected effect corresponding to the input information, the first processing model does not need to be invoked multiple times, and consumption of the first processing model is reduced. A target to-be-invoked object is determined based on the task list, an object parameter corresponding to the task description information is determined through a second processing model associated with the target to-be-invoked object, the task description information is translated into the object parameter through the second processing model associated with the to-be-invoked object, the object parameter does not need to be determined through the first processing model, so that the context information of the first processing model is reduced, and the number of to-be-invoked objects is facilitated to be expanded. The output content of the task is determined based on the object parameter through the target to-be-invoked object, and target media content is generated based on the output content of the target to-be-invoked object. The embodiments of the present disclosure combine the first processing model and the second processing model to quickly complete task arrangement and object invocation, shorten the time consumption of the generation process of the media content, and can access more model capabilities, so as to realize accurate generation and rapid presentation of media content effects

[0064] Figure 2 For another flowchart of a media content generation method provided by the embodiments of the present disclosure, the embodiments of the present disclosure are based on the above embodiments, and specifically limit the determination of a target to-be-invoked object based on the task list. As shown in Figure 2 The method includes:

[0065] S210, obtaining input information, wherein the input information includes text.

[0066] S220, determining a task list based on the input information through a first processing model, wherein the task list includes task information of multiple tasks, and the task information includes identification information of a to-be-invoked object and task description information.

[0067] S230, determining a calling sequence of tasks in the task list based on input content information and output content information of the tasks in the task list, wherein the calling sequence includes serial or parallel.

[0068] The calling sequence represents the order of calling the tasks in the task list. Serial means that for a task whose output content information of another task is used as its input content information, the task needs to wait for the execution of the preceding task to be completed before being called. Parallel means that for a task that does not depend on the output content information of another task or the input content information of the task has been generated, the task can be called.

[0069] Since the input content information of a task represents the input data on which the task execution depends, and the output content information represents the result data generated by the completion of the task execution, if the output content information of a task is the input content information of another task, it can be determined that the latter task depends on the execution result of the former task, and the two tasks need to be called in series. If there is no input dependency between the two tasks, the two tasks can be called in parallel.

[0070] For example, by traversing the input content information and the output content information of the tasks in the task list, a plurality of target tasks that do not require input content information or for which the input content information has been generated are determined, and the plurality of target tasks are called in parallel. For other tasks that require input content information and for which the input content information has not been generated, the other tasks are called after the input content information of the other tasks is generated.

[0071] S240, determining a target to-be-called object based on the calling sequence.

[0072] The target to-be-called object represents a to-be-called object corresponding to the target task. The target task refers to a task in the task list that can be currently executed. If the current task in the task list does not require input content information or the input content information has been generated, the current task is determined as the target task. Based on the identification information of the to-be-called object in the task information of the target task, the target to-be-called object is determined.

[0073] S250, determining an object parameter corresponding to the task description information through a second processing model associated with the target to-be-called object, and determining the output content corresponding to the task through the target to-be-called object based on the object parameter.

[0074] For example, the task description information corresponding to the target task is sent to the second processing model corresponding to the target to-be-called object, and the service parameter corresponding to the task description information is determined based on the second processing model based on preset prompt information, and the service parameter is sent to the target to-be-called object. The preset prompt information is constraint information on the object parameter of the target to-be-called object. For example, the preset prompt information includes the composition requirement of the prompt word of the target to-be-called object. The output content of the target to-be-called object is obtained by executing the target task based on the object parameter through the target to-be-called object.

[0075] S260, generating target media content based on the output content of the target to-be-called object.

[0076] For example, the output content includes output text and media material, and the media material includes output pictures and / or output videos. The output text can be summary content, and the output pictures and / or output videos are connected through the output text, so that the transition between the pictures and / or videos is smoother and more natural.

[0077] Figure 3 Another flowchart of a media content generation method is provided by the embodiments of the present disclosure, as shown in Figure 3 As shown, input information 310 is obtained, the input information 310 is input into a first processing model 320, the intention attribute of the input information 310 is understood through the first processing model 320, and a natural language is used to split the planning intention implementation process to obtain a task list 330 including multiple tasks. The tasks correspond to the steps split from the intention attribute implementation process of the input information. The task list 330 includes task information corresponding to the multiple tasks, and the task information includes the name of a to-be-called object, task description information, input content information, and output content information. Optionally, the to-be-called object includes a text-to-image tool 340, a text generation tool 350, and an image-to-image tool 360. The calling order of the tasks in the task list is determined based on the input content information and the output content information, and a target task is determined based on the calling order. The identification information of the to-be-called object in the task information of the target task determines a target to-be-called object. The task description information of the target task is sent to a second processing model 390 associated with the target to-be-called object, the task description information is translated into object parameters of the target to-be-called object through the second processing model 390, and the object parameters are output to the target to-be-called object. The target to-be-called object executes the target task based on the object parameters, and output content of the target to-be-called object is obtained. The output content includes output text 360 and output image 370, etc. The output text 360 and the output image 370 are arranged based on the order of the tasks in the task list, and target media content 380 is obtained.

[0078] The technical scheme of the embodiments of the present disclosure determines a task list based on input information through a first processing model, determines the calling order of the tasks in the task list based on the input content information and the output content information of the tasks in the task list, determines a target to-be-called object based on the calling order, and the calling order includes serial or parallel. Through parallel calling of multiple target to-be-called objects, the generation time consumption of target media content can be shortened. Through the structure that the to-be-called object cooperates with the second processing model, the task description information is converted into object parameters of the to-be-called object, and the target to-be-called object executes the task based on the object parameters, which helps to access more model capabilities in the model architecture.

[0079] Figure 4 A structural diagram of a media content generation device provided by the embodiments of the present disclosure is shown. The device can be implemented in the form of software and / or hardware, and can be implemented by an electronic device such as a mobile terminal, a PC terminal, or a server.

[0080] As shown in Figure 4 The device includes an information obtaining module 410, a task determining module 420, an object calling module 430, and a content generation module 440.

[0081] The information obtaining module 410 is configured to obtain input information, wherein the input information comprises text.

[0082] The task determining module 420 is configured to determine a task list based on the input information by using a first processing model, wherein the task list comprises task information of a plurality of tasks, and the task information comprises identification information of a to-be-invoked object and task description information.

[0083] The object invoking module 430 is configured to determine a target to-be-invoked object based on the task list, determine an object parameter corresponding to the task description information by using a second processing model associated with the target to-be-invoked object, and determine output content corresponding to the task based on the object parameter by using the target to-be-invoked object.

[0084] The content generating module 440 is configured to generate target media content based on the output content of the target to-be-invoked object.

[0085] Optionally, the input information further comprises at least one of voice, a picture, and a video.

[0086] Optionally, the task information further comprises input content information and output content information, and the object invoking module 430 is specifically configured to:

[0087] determine a calling sequence of the tasks in the task list based on the input content information and the output content information of the tasks in the task list, wherein the calling sequence comprises serial or parallel;

[0088] determine the target to-be-invoked object based on the calling sequence.

[0089] Optionally, the determining of the calling sequence of the tasks in the task list based on the input content information and the output content information of the tasks in the task list comprises:

[0090] determining a dependency relationship of the tasks in the task list based on the input content information and the output content information of the tasks in the task list;

[0091] parallelly invoking to-be-invoked objects corresponding to at least two tasks in the task list that do not have a dependency relationship;

[0092] serially invoking to-be-invoked objects corresponding to at least two tasks in the task list that have a dependency relationship.

[0093] Optionally, the task determining module 420 is specifically configured to:

[0094] The input information is input into the first processing model, an intention attribute of the input information is determined through the first processing model, a plurality of tasks are generated based on the intention attribute, and a task list corresponding to the tasks is output.

[0095] Optionally, the object calling module 430 is further configured to:

[0096] sending the task description information to a second processing model associated with the target object to be called;

[0097] converting the task description information into the object parameter based on preset prompt information through the second processing model, and outputting the object parameter to the object to be called, wherein the preset prompt information includes constraint information for generating the object parameter of the object to be called.

[0098] Optionally, the content generation module 440 is specifically configured to:

[0099] obtaining output text and media material returned by the object to be called, wherein the media material includes output pictures and / or output videos;

[0100] arranging the output text and the media material based on the order of the tasks in the task list to obtain the target media content.

[0101] The media content generation apparatus provided in the embodiments of the present disclosure can perform the media content generation method provided in any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of performing the method.

[0102] It should be noted that each unit and module included in the apparatus is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific name of each functional unit is only for convenient mutual distinction, and does not serve to limit the protection scope of the embodiments of the present disclosure.

[0103] Figure 5 A structural schematic diagram of an electronic device provided in the embodiments of the present disclosure is shown in FIG. 5. Hereinafter, the structural schematic diagram of the electronic device (for example, a terminal device or a server) 500 suitable for implementing the embodiments of the present disclosure is shown in FIG. 5. Figure 5 Figure 5 The terminal device in the embodiments of the present disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a vehicle-mounted terminal (for example, a vehicle-mounted navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. Figure 5 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.​

[0104] like Figure 5 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0105] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0106] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0107] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0108] The electronic device provided in this embodiment and the media content generation method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0109] The embodiment of the present disclosure provides a computer storage medium, which stores a computer program, and the computer program is executed by a processor to implement the media content generation method provided by the above embodiment.

[0110] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination thereof.

[0111] In some embodiments, the client, server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0112] The above computer readable medium can be included in the above electronic device; or can exist separately without being assembled into the electronic device.

[0113] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to:

[0114] Obtain input information, wherein the input information includes text;

[0115] Determine a task list based on the input information through a first processing model, wherein the task list includes task information of a plurality of tasks, and the task information includes identification information of a to-be-invoked object and task description information;

[0116] Determine a target to-be-invoked object based on the task list, determine an object parameter corresponding to the task description information through a second processing model associated with the target to-be-invoked object, and determine output content corresponding to the task based on the object parameter through the target to-be-invoked object;

[0117] Generate target media content based on the output content of the target to-be-invoked object.

[0118] Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0119] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The computer program product of the first aspect can include a non-transitory computer-readable medium storing code that, when executed, causes a computer to perform operations for the first aspect.

[0120] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0121] The functions described in this description above can be implemented in hardware, software, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media include both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, functional

[0122] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium can include a tangible, non-transitory memory such as a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] The above description merely illustrates the preferred embodiments of the disclosure and a principle for applying the technologies. It is understood by those skilled in the art that the disclosed scope of the disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the disclosure (but not limited to) can be formed.

[0124] Further, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated or sequentially. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0125] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for generating media content, characterized in that, include: Obtain input information, wherein the input information includes text; The first processing model determines a task list based on the input information, wherein the task list includes task information for multiple tasks, and the task information includes the identification information of the object to be invoked and the task description information. Based on the task list, a target object to be invoked is determined. The object parameters corresponding to the task description information are determined through the second processing model associated with the target object to be invoked. The output content corresponding to the task is determined based on the object parameters of the target object to be invoked. Target media content is generated based on the output content of the target object to be invoked.

2. The method according to claim 1, characterized in that, The input information also includes at least one of voice, images, and videos.

3. The method according to claim 1, characterized in that, The task information also includes input content information and output content information. The step of determining the target object to be invoked based on the task list includes: Based on the input and output information of the tasks in the task list, the calling order of the tasks in the task list is determined, wherein the calling order includes serial or parallel. The target object to be called is determined based on the calling order.

4. The method according to claim 3, characterized in that, The step of determining the invocation order of tasks in the task list based on the input and output information of the tasks in the task list includes: Based on the input and output information of the tasks in the task list, the dependencies between the tasks in the task list are determined. In parallel, invoke the invocation objects corresponding to at least two tasks in the task list that do not have any dependencies; The task list is serially invoked to the corresponding objects of at least two tasks that have dependencies on each other.

5. The method according to claim 1, characterized in that, The step of determining the task list based on the input information using the first processing model includes: The input information is input into the first processing model, the intent attribute of the input information is determined by the first processing model, multiple tasks are generated based on the intent attribute, and a task list corresponding to the task is output.

6. The method according to claim 1, characterized in that, The step of determining the object parameters corresponding to the task description information through the second processing model associated with the target object to be invoked includes: The task description information is sent to the second processing model associated with the target object to be invoked. The second processing model converts the task description information into the object parameters based on preset prompt information, and outputs the object parameters to the object to be called. The preset prompt information includes constraint information for generating the object parameters of the object to be called.

7. The method according to claim 1, characterized in that, The process of generating target media content based on the output content of the target object to be invoked includes: Obtain the output text and media materials returned by the object to be invoked, wherein the media materials include output images and / or output videos; Based on the order of the tasks in the task list, the output text and media materials are arranged to obtain the target media content.

8. A media content generation device, characterized in that, include: An information acquisition module is used to acquire input information, wherein the input information includes text; The task determination module is used to determine a task list based on the input information through a first processing model, wherein the task list includes task information of multiple tasks, and the task information includes the identification information of the object to be invoked and the task description information; The object invocation module is used to determine the target invocation object based on the task list, determine the object parameters corresponding to the task description information through the second processing model associated with the target invocation object, and determine the output content corresponding to the task based on the object parameters through the target invocation object. The content generation module is used to generate target media content based on the output content of the target object to be invoked.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the media content generation method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the media content generation method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the media content generation method as described in any one of claims 1-7.

Images