Video generation method and device based on multi-agent cooperation and agents

Abstract

The invention provides a video generation method and device based on multi-agent cooperation and agents, relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, large models, agents, AIGC, element universe and the like, and is applied to application scenes such as wisdom education, video production, animation production and the like. The method comprises the following steps: executing a target sub-task in a target task by utilizing at least one target agent to obtain a target sub-execution result; and generating a target video based on the target sub-execution result, the target agent executing the following operations: obtaining sub-task demand information for the target sub-task; determining a subtask execution element for controlling the execution process of the target subtask by using a first subagent of the target agent based on the subtask demand information; and utilizing a second sub-agent of the target agent to execute the target sub-task based on the sub-task execution element to obtain a target sub-execution result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as deep learning, large models, intelligent agents, AIGC (Artificial Intelligence Generated Content), and the metaverse, and is applied to application scenarios such as intelligent education, video production, and animation production. Background Art

[0002] With the rapid development of artificial intelligence technology, video data can be generated based on large models with generative capabilities such as multi-modal large models to meet the needs of users for quickly producing video data. Summary of the Invention

[0003] The present disclosure provides a video generation method, apparatus, intelligent agent, electronic device, and storage medium based on multi-agent collaboration.

[0004] According to one aspect of the present disclosure, there is provided a video generation method based on multi-agent collaboration, including: using at least one target intelligent agent to execute a target sub-task in a target task to obtain a target sub-execution result; and generating a target video based on the target sub-execution result, where the target intelligent agent performs the following operations: obtaining sub-task requirement information for the target sub-task; using a first sub-intelligent agent of the target intelligent agent to determine sub-task execution elements for controlling the execution process of the target sub-task based on the sub-task requirement information; using a second sub-intelligent agent of the target intelligent agent to execute the target sub-task based on the sub-task execution elements to obtain a target sub-execution result.

[0005] According to another aspect of the present disclosure, there is provided a video generation apparatus based on multi-agent collaboration, including: an execution module for using at least one target intelligent agent to execute a target sub-task in a target task to obtain a target sub-execution result; and a target video generation module for generating a target video based on the target sub-execution result, where the target intelligent agent performs the following operations: obtaining sub-task requirement information for the target sub-task; using a first sub-intelligent agent of the target intelligent agent to determine sub-task execution elements for controlling the execution process of the target sub-task based on the sub-task requirement information; using a second sub-intelligent agent of the target intelligent agent to execute the target sub-task based on the sub-task execution elements to obtain a target sub-execution result.

[0006] According to another aspect of the present disclosure, there is also provided an intelligent agent of artificial intelligence configured to execute a video generation method based on multi-agent collaboration.

[0007] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a video generation method based on multi-agent collaboration.

[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute a video generation method based on multi-agent collaboration.

[0009] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program which, when executed by a processor, implements a video generation method based on multi-agent collaboration.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0012] Figure 1 Schematically shows an exemplary system architecture to which a video generation method and apparatus based on multi-agent collaboration can be applied according to an embodiment of the present disclosure;

[0013] Figure 2A Schematically shows a flowchart of a video generation method based on multi-agent collaboration according to an embodiment of the present disclosure;

[0014] Figure 2B Schematically shows a flowchart of using at least one target agent to execute a target subtask in a target task according to an embodiment of the present disclosure.

[0015] Figure 3 Schematically shows a schematic diagram of using a target agent to execute a target subtask according to an embodiment of the present disclosure;

[0016] Figure 4 Schematically shows a schematic diagram of using a designated agent to generate a task scheduling result according to an embodiment of the present disclosure;

[0017] Figure 5 Schematically shows a schematic diagram of using a target agent to execute a target subtask according to an embodiment of the present disclosure;

[0018] Figure 6Schematically shows a schematic diagram of a video generation method based on multi-agent collaboration according to an embodiment of the present disclosure;

[0019] Figure 7A Schematically shows a schematic diagram of the effect of the target video according to an embodiment of the present disclosure;

[0020] Figure 7B Schematically shows a schematic diagram of the effect of the health science popularization video according to an embodiment of the present disclosure;

[0021] Figure 7C Schematically shows a schematic diagram of the effect of the legal popularization video according to an embodiment of the present disclosure;

[0022] Figure 8 Schematically shows a block diagram of a video generation device based on multi-agent collaboration according to an embodiment of the present disclosure;

[0023] Figure 9 Schematically shows a block diagram of the structure of an agent of artificial intelligence according to an embodiment of the present disclosure; and

[0024] Figure 10 Shows a schematic block diagram of an example electronic device that can be used to implement the video generation method based on multi-agent collaboration of the embodiments of the present disclosure. Detailed implementation manners

[0025] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and public order and good customs are not violated.

[0027] The inventors found that with the rapid development of video platforms, more and more users produce video products based on personal needs. However, the video production process usually takes a long time, and related video production tools are difficult to accurately generate video data that meets the user's needs, or the generated video quality is low, making it difficult to meet the actual needs of users.

[0028] The present disclosure provides a video generation method, apparatus, agent, electronic device, and storage medium based on multi-agent collaboration. The video generation method based on multi-agent collaboration includes: using at least one target agent to execute a target subtask in a target task to obtain a target sub-execution result; and generating a target video based on the target sub-execution result, where the target agent performs the following operations:

[0029] Obtain subtask requirement information for the target subtask; use a first sub-agent of the target agent to determine subtask execution elements for controlling the execution process of the target subtask based on the subtask requirement information; use a second sub-agent of the target agent to execute the target subtask based on the subtask execution elements to obtain a target sub-execution result.

[0030] According to an embodiment of the present disclosure, by using the first sub-agent in the target agent to determine the subtask execution elements based on the subtask requirement information for the target subtask, the first agent can analyze the subtask requirement information more accurately, and the subtask execution elements can more accurately represent the execution process of the target subtask, so that the target subtask can be executed according to the requirement attributes characterized by the subtask requirement information. Furthermore, by using the second sub-agent to execute the target subtask based on the subtask execution elements, the second sub-agent can be more precisely controlled to execute the target subtask according to the requirement attributes characterized by the subtask requirement information, so that the obtained target sub-execution result conforms to the requirement attributes represented by the subtask requirement information. In this way, a target video that can more accurately meet the user's needs can be generated based on the target sub-execution results obtained by at least one target agent executing the target subtask, improving the matching degree between the target video and the user's needs and improving the quality of the target video.

[0031] Figure 1 Schematically shows an exemplary system architecture to which the video generation method and apparatus based on multi-agent collaboration according to an embodiment of the present disclosure can be applied.

[0032] It should be noted that Figure 1 The illustration is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments, or scenarios. For example, in another embodiment, the exemplary system architecture to which the video generation method and apparatus based on multi-agent collaboration can be applied may include a terminal device, but the terminal device can implement the video generation method and apparatus based on multi-agent collaboration provided by the embodiments of the present disclosure without interacting with a server.

[0033] Such as Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0034] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0035] The terminal devices 101, 102, 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, and desktop computers, etc.

[0036] The server 105 may be a server that provides various services, such as a background management server that supports the content browsed by users using the terminal devices 101, 102, 103 (for example only). The background management server may analyze and process data such as received user requests, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0037] The server 105 may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server 105 may also be a server of a distributed system, or a server combined with blockchain.

[0038] The server 105 can obtain a target intelligent agent by deploying a generative model. It should be noted that the first sub-intelligent agent and the second sub-intelligent agent in the target intelligent agent may be deployed in the same server, or the first sub-intelligent agent and the second sub-intelligent agent in the target intelligent agent may also be deployed in different servers.

[0039] It should be noted that the video generation method based on multi-agent collaboration provided by the embodiments of the present disclosure can generally be executed by the server 105. Correspondingly, the video generation device based on multi-agent collaboration provided by the embodiments of the present disclosure can generally be set in the server 105. The video generation method based on multi-agent collaboration provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103, and / or the server 105. Correspondingly, the video generation device based on multi-agent collaboration provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103, and / or the server 105.

[0040] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0041] Figure 2A are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0042] As Figure 2A shown, the video generation method based on multi-agent collaboration includes operations S210 to S220.

[0043] In operation S210, at least one target agent is used to execute the target subtask in the target task to obtain a target sub-execution result.

[0044] In operation S220, a target video is generated based on the target sub-execution result.

[0045] According to an embodiment of the present disclosure, the target agent can be an execution entity for executing the target subtask constructed based on a large model with a generation function such as a generative model. The target agent can be deployed in a server, or can also be separately deployed in multiple servers connected by communication.

[0046] According to an embodiment of the present disclosure, the target task can be used to generate a target video. The target task can include one or more target subtasks, and the target subtask can be used to generate an intermediate result for determining the target video. The target subtask can be a processing process including multiple execution steps, for example, it can involve intelligent reasoning links such as language understanding, context analysis, and logical reasoning. The target subtask can include any task type, such as a copywriting generation type, a speech synthesis type, etc.

[0047] According to an embodiment of the present disclosure, the target sub-execution result may include intermediate results for generating a video, such as a video copywriting generation result, an image rendering result, a synthesized voice data, and so on. Generating a target video based on the target sub-execution result as video material may include performing video rendering based on one or more target sub-execution results to obtain the target video.

[0048] In one example, a vision large model may be used to process the video copywriting and multi-frame images in the target sub-execution result to obtain the target video.

[0049] In one example, the target sub-execution result may be synthesized voice data and explanatory copywriting, and the synthesized voice data and explanatory copywriting may be filled into a video compilation script template so that the updated video compilation script can be rendered to obtain the target video.

[0050] In one example, one or more target sub-execution results may also be processed by an intelligent agent with video generation capabilities to generate the target video.

[0051] It should be noted that the embodiments of the present disclosure do not limit the specific manner of generating the target video, as long as it is based on at least one target sub-execution result generated by the target intelligent agent.

[0052] According to an embodiment of the present disclosure, the target intelligent agent may include multiple sub-intelligent agents. The sub-intelligent agent may be an intelligent agent with an independent task execution function, and the target intelligent agent may be a system including multiple sub-intelligent agents. By using multiple sub-intelligent agents to cooperate to execute the target sub-task, the execution efficiency of the target sub-task and the accuracy and generation effect of the target sub-execution result can be improved.

[0053] Figure 2B Schematically shows a flowchart of using at least one target intelligent agent to execute a target sub-task in a target task according to an embodiment of the present disclosure.

[0054] As Figure 2B shown, the target intelligent agent performs the following operations S211 to operation S213 to execute the target sub-task.

[0055] In operation S211, obtain sub-task requirement information for the target sub-task.

[0056] In operation S212, use the first sub-intelligent agent of the target intelligent agent to determine sub-task execution elements for controlling the execution process of the target sub-task based on the sub-task requirement information.

[0057] In operation S213, use the second sub-intelligent agent of the target intelligent agent to execute the target sub-task based on the sub-task execution elements to obtain the target sub-execution result.

[0058] According to an embodiment of the present disclosure, the sub-task requirement information for a target sub-task may be used to represent the requirement intention of a target object for a target video. For example, it may represent requirement intentions such as the duration, style, and theme of the target video. The sub-task requirement information may represent the requirement intentions that a target agent executing the target sub-task needs to pay attention to. For example, a copywriting generation agent executing a copywriting generation sub-task needs to pay attention to requirement intentions such as the video theme and video duration.

[0059] The execution entity for obtaining the sub-task requirement information may be any sub-agent in the target agent. For example, the first sub-agent may be used as the execution entity to obtain the sub-task requirement information. However, it is not limited to this. The sub-task requirement information may also be obtained based on other sub-agents or functional modules included in the target agent and transmitted to the first sub-agent. The embodiments of the present disclosure do not limit the specific execution entity of step S211, as long as it is a functional module included in the target agent.

[0060] According to an embodiment of the present disclosure, the sub-task requirement information may include some or all of the requirement intentions. In the case where the sub-task requirement information includes all the requirement intentions, the first sub-agent may be used to perform a full-scale analysis of the requirement intentions of the target object to improve the matching degree between the sub-task execution elements and the requirement intentions, and to avoid the absence of intention in the sub-task execution elements or contradictions with the requirement intentions.

[0061] According to an embodiment of the present disclosure, the first sub-agent and the second sub-agent may be two different agents included in the target agent. The first sub-agent may perform semantic analysis based on the sub-task requirement information and generate sub-task execution elements for controlling the execution process of the target sub-task in combination with the task type of the target sub-task. The sub-task execution elements may control specific execution processes such as the execution steps, execution parameters, execution logic, and information acquisition of the target sub-task. In this way, the second sub-agent with the function of executing the target sub-task can execute the target sub-task more precisely according to the requirement intention under the control of the sub-task execution elements, so that the target sub-execution result matches the requirement intention of the target object, and to avoid conflicts between the sub-task execution result and the requirement intention, resulting in the display effect of the target video not meeting the actual needs of the user.

[0062] In one example, the sub-task execution elements may include control information such as an execution code script and task configuration parameters for controlling the second sub-agent to execute the target sub-task. So that the second sub-agent can accurately execute the script based on the task configuration parameters to generate a target sub-execution result that meets the requirement intention.

[0063] According to an embodiment of the present disclosure, by decomposing the target task of generating a target video into target subtasks, analyzing the subtask requirement information through a first sub-agent in the target agent to obtain subtask execution elements for controlling the execution process of the target subtask, and using a second sub-agent to accurately execute the target subtask based on the control of the subtask execution elements, it is possible to achieve cooperation among multiple sub-agents to finely control the execution process of the target subtask, improve the execution efficiency of the target subtask, and improve the matching degree between the target execution result and the demand intention of the target object. Furthermore, by finely controlling the quality of the target execution result, the video quality of the target video can be improved to meet the video generation requirements of the target object.

[0064] According to an embodiment of the present disclosure, the subtask execution elements include at least one of a style consistency condition and a logical coherence condition.

[0065] According to an embodiment of the present disclosure, the style consistency condition indicates that the expression style of the target execution result matches the demand style represented by the object demand information. The target execution result can include any type such as text, speech, video frame, etc. The expression style includes at least one of a text expression style, a speech expression style, and a video frame expression style.

[0066] The text expression style can be any type of expression style such as lively, serious, colloquial, dialect style, etc. The speech expression style can include the same style types as the text expression style such as lively and serious. In addition, the speech expression style can also include styles related to speech attributes such as gender and tone. The video frame style can represent the image style of the video background image, video illustration, video frame image, etc. shown in the video. The video frame style can be related to

[0067] It should be noted that the expression style can be represented based on tags. By training multiple sub-agents in the target agent using sample data labeled with style tags, the sub-agents can learn the relationship between the subtask execution results of the corresponding task types and the style tags. Thus, the subtask execution elements output by the first sub-agent can represent the expression style corresponding to the subtask requirement information, and the second sub-agent can accurately execute generation tasks such as copywriting generation and image generation based on the style consistency condition.

[0068] According to an embodiment of the present disclosure, the logical coherence condition characterizes that there is logical coherence among multiple target sub - execution results of a target task. The target task may include multiple target subtasks, and the multiple target subtasks may be collaboratively executed by different multiple target agents to obtain multiple target sub - execution results. The logical coherence among the multiple target sub - execution results may indicate that different target sub - execution results such as text, images, and voices express the same theme and requirement intention. Or the logical coherence may indicate that the multiple target sub - execution results have the same or similar logical structures, avoiding logical confusion among the multiple target sub - execution results.

[0069] For example, if the multiple target sub - execution results are a video copywriting and a video illustration respectively, and the video copywriting represents the text content of the prevention method for diabetes, while the video illustration shows a picture of a pet cat, it can be considered that there is no logical coherence between the video copywriting and the video illustration.

[0070] Again, for example, if the multiple target sub - execution results are a video copywriting and a video subtitle respectively, and the operation execution order between the multiple operation steps represented by the video copywriting for power equipment is different from that represented by the video subtitle, it can be considered that there is no logical coherence between the video copywriting and the video illustration.

[0071] It should be noted that the subtask execution elements may include any type of information such as subtask requirement conditions, requirement condition explanation texts, requirement condition examples, etc. The subtask execution elements can be used as prompt information for the second sub - agent to execute the target subtask, so as to control the second sub - agent to execute the target subtask.

[0072] According to an embodiment of the present disclosure, the target subtask includes at least one of the following: a copywriting generation subtask, a storyboard data generation subtask, a graphic element generation subtask, and a subtitle generation subtask.

[0073] The second sub - agent can execute the copywriting generation subtask based on the copywriting generation subtask execution elements for the copywriting generation subtask to obtain a target copywriting that matches the requirement intention of the target object. The target copywriting can be used as a component element of the target video, such as the explanatory voice of the target video, inserted text, etc. The copywriting generation subtask execution elements may include text theme conditions, text word count requirement conditions, text expression style requirement conditions, and so on.

[0074] In one example, the target copywriting can be used to generate voice data, so that the target video can introduce product information, equipment implementation steps and other copywriting contents through the target copywriting represented by the voice data.

[0075] Figure 3 Schematically shows the schematic diagram of using the target agent to execute the target subtask according to an embodiment of the present disclosure.

[0076] AsFigure 3 As shown, the target agent 310 for performing the copywriting generation subtask may include a first sub-agent 311 and a second sub-agent 312. The first sub-agent 311 obtains the requirement intention 301 as the subtask requirement information, and performs semantic analysis on the requirement intention 301 "Disease A, prevention, long text, step by step" to obtain the subtask execution elements for the copywriting generation subtask. The second sub-agent 312 executes the copywriting generation subtask based on the subtask execution elements to obtain the target copywriting 302 as the target sub-execution result. The second sub-agent 312 controls the execution process of the copywriting generation subtask based on the subtask execution elements, and can generate multiple step texts including step identifiers, and the target copywriting 302 is also a long text representing the prevention of Disease A.

[0077] The storyboard data generation subtask is used to generate the target storyboard data. The target storyboard data can indicate the switching timing of different video frame images in the target video, the display timing, display duration, etc. of video elements such as images and texts inserted in the target video. The storyboard data generation subtask execution elements for the storyboard data generation subtask may include constraints on the display attributes such as the display timing and display duration of video elements, or the storyboard data generation subtask execution elements may also indicate that the display attributes of video elements are adapted to the voice content in the target video. For example, the storyboard data generation subtask execution elements may indicate that the voice content "Eat less high-sugar foods" has the same display timing as the image element representing the prohibition of eating chocolate.

[0078] According to an embodiment of the present disclosure, the second sub-agent may generate graphic elements by executing the graphic element generation subtask. The graphic elements may include content represented based on image elements such as the background image, inserted image, inserted animation, inserted word art, etc. of the video. The graphic element generation subtask execution elements for the graphic element generation subtask may represent requirement conditions related to graphic elements such as the image content classification, image element style, graphic element size, graphic element arrangement method, etc. of the graphic elements to be generated.

[0079] It should be noted that the second sub-agent may generate graphic elements based on the image generation ability of the vision large model, or may also perform intelligent enhanced retrieval based on semantic understanding ability to retrieve graphic elements. The embodiments of the present disclosure do not limit the specific execution manner of the second sub-agent for executing the graphic element generation subtask.

[0080] According to an embodiment of the present disclosure, the subtitle generation subtask can be understood as generating target subtitles in a target video by a second sub-execution entity according to the subtitle generation subtask execution elements. The target subtitles can include subtitle text content, timestamps of subtitle text, and text characters in the subtitle text that need to be displayed in emphasized display modes such as bold and underline. The subtitle generation subtask execution elements can be output by a first sub-agent based on the understanding and analysis of the subtask requirement information, so as to control the second sub-agent to accurately execute the subtitle generation subtask based on the subtitle generation subtask execution elements. In addition, the subtitle generation subtask execution elements can also include at least one of the above-mentioned subtask requirement conditions such as the logical coherence condition and the expression style consistency condition, so as to control the logic and expression between the target subtitles and other target sub-execution results to be consistent.

[0081] According to an embodiment of the present disclosure, obtaining the subtask requirement information for a target subtask includes: using a target agent to perform requirement information detection on the object requirement information of a target object based on the subtask type of the target subtask to obtain the subtask requirement information.

[0082] According to an embodiment of the present disclosure, the subtask type can include any subtask type such as a copywriting generation type, a graphic element generation type, etc. The first sub-agent can be used to perform requirement information detection on the object requirement information of the target object based on the subtask type of the target subtask to obtain the subtask requirement information related to the subtask type. Or, other sub-agents or functional modules in the target agent can also be used to perform requirement information detection on the object requirement information of the target object to obtain the subtask requirement information.

[0083] According to an embodiment of the present disclosure, by using a target agent to perform requirement information detection on object requirement information based on the subtask type to obtain the subtask requirement information for a target subtask, the subtask requirement information can be matched with the execution process of the target subtask with the subtask type, so as to realize the preliminary understanding of the requirement intention of the object requirement information based on the target agent, so that the target agent can learn the requirement intention of the target object more comprehensively, and the subtask execution elements generated by the subsequent first sub-agent based on the subtask requirement information can more accurately represent the requirement intention of the target object, so that the subtask execution elements can closely relate the execution process of the target subtask with the requirement intention, and improve the quality of the target sub-execution result.

[0084] According to an embodiment of the present disclosure, the target task includes a plurality of target subtasks. The plurality of target subtasks can be assigned to a plurality of target agents for execution, or the plurality of target subtasks can be assigned to a target agent and a non-agent functional module for executing the target subtask for execution. The specific assignment method of the plurality of target subtasks in the embodiment of the present disclosure is not limited.

[0085] According to an embodiment of the present disclosure, the video generation method based on multi-agent collaboration further includes using a designated agent to perform the following operations: performing intent recognition on the object requirement information of the target object to obtain a requirement intent; determining a plurality of target subtasks and the dependency relationships between the plurality of target subtasks based on the requirement intent; and sending the plurality of target subtasks and the dependency relationships between the plurality of target subtasks to a plurality of target agents.

[0086] According to an embodiment of the present disclosure, the designated agent may be any one of the plurality of target agents, or the designated agent may also be an agent that is different from the target agent used to execute the target subtask.

[0087] According to an embodiment of the present disclosure, as an execution subject, the designated agent can perform intent recognition by semantically understanding the object requirement information, so as to determine a plurality of subtask types that can meet the requirement intent based on the requirement intent represented by the intent recognition result, and then determine a plurality of target subtasks, thereby realizing the mapping from the requirement intent to a plurality of specific subtasks. The designated agent can also determine the execution logic between the plurality of target subtasks based on the specific execution elements of the target task for generating the target video, and then obtain the dependency relationships between the plurality of target subtasks. By sending the target subtasks corresponding to the target agents and the dependency relationships between the plurality of target subtasks to the plurality of target agents, the plurality of target agents can each obtain the associated sub-execution results from the associated target agents with dependency relationships based on the dependency relationships, and then automatically execute the target subtasks based on the execution logic of the plurality of target subtasks, improving the execution efficiency of the target task.

[0088] According to an embodiment of the present disclosure, the dependency relationships between the plurality of target subtasks can be represented based on a topological structure representing the execution logic of the target subtasks. Alternatively, the dependency relationships between the plurality of target subtasks can also be represented in other forms such as a flowchart, and the embodiments of the present disclosure do not limit this.

[0089] In one example, the specified agent is a task orchestration agent that is different from the target agent for executing the target subtask. The task orchestration agent can determine the target task for generating the target video according to the demand intention of the target object by performing intention recognition on the object demand information, and split the target task into multiple target subtasks according to the task elements of the target task. The task orchestration agent distributes multiple target subtasks and dependencies to the target agents corresponding to the subtask types respectively through the corresponding subtask types of the target agents, so as to realize the task orchestration of multiple target agents, and based on the dependencies, enable the target agents to transmit the target sub-execution results generated by the target agents to the next target agent according to the execution logic of the target task, realizing the automatic intelligent transfer of the target sub-execution results of the target subtasks among multiple target agents, so as to improve the execution efficiency of the target task.

[0090] According to an embodiment of the present disclosure, the subtask demand information for the target subtask is determined by the specified agent based on the demand intention.

[0091] According to an embodiment of the present disclosure, the demand intention can be used as the subtask demand information, and the specified agent sends the subtask demand information to the target agent, or the target agent obtains the subtask demand information from the specified agent. By sending the demand intention to each target agent as the subtask demand information, the target agent can reduce the process of performing intention recognition on the object demand information and reduce the computational overhead generated by the target agent. At the same time, different target agents may have semantic conflicts or semantic losses in the demand intentions represented by the subtask demand information obtained by their respective intention recognition of the same object demand information, resulting in the mismatch between the target sub-execution result and the demand of the target object. Therefore, by sending the demand intention to multiple target agents as the subtask demand information, multiple target agents can execute their respective target subtasks based on the same or similar demand intentions, realizing the consistency and logical coherence among multiple target sub-execution results. Figure 1 Consistency and logical coherence.

[0092] It should be noted that the subtask demand information sent to the target agent can be user-specific subtask demand for representing the demand intention, and the target agent can also obtain the basic subtask demand information required for the target subtask of the subtask type. The basic subtask demand information can be, for example, text coherence requirement conditions, sensitive word avoidance requirement conditions, and so on.

[0093] According to an embodiment of the present disclosure, determining a plurality of target subtasks based on a demand intention, and the dependency relationships between the plurality of target subtasks include: using a first designated sub-agent of a designated agent to perform a task orchestration task based on the demand intention, obtaining a plurality of initial subtasks and initial dependency relationships; using a second designated sub-agent of the designated agent to perform a task orchestration detection subtask on the plurality of initial subtasks and initial dependency relationships based on the video demand conditions represented by the demand intention, obtaining a task orchestration detection result; in the case where the task orchestration detection result does not meet the video demand conditions, using the first designated sub-agent to perform a task orchestration subtask based on the demand intention and the task orchestration detection result, obtaining a plurality of target subtasks that meet the video demand conditions, and the dependency relationships between the plurality of target subtasks.

[0094] According to an embodiment of the present disclosure, the first designated sub-agent and the second designated sub-agent can be agents for performing different generation functions. The first designated sub-agent generates a plurality of initial subtasks that can match the demand intention and the initial dependency relationships between the plurality of initial subtasks based on the understanding of the demand intention. The demand intention can, for example, represent video demand conditions such as the video length demand, video theme demand, and video style demand for the target video. By using the second designated sub-agent to perform a task orchestration detection subtask on the plurality of initial subtasks and initial dependency relationships with the video demand conditions as hint information, it is possible to use the second designated sub-agent to detect the subtask types of the initial subtasks and whether the plurality of initial dependency relationships can meet the video demand conditions, and based on the defect types characterized by the task orchestration detection result, to indicate which specific video demand conditions the initial subtasks and initial dependency relationships do not meet. Thus, in the case where the task orchestration detection result does not meet the video demand conditions, by using the first designated sub-agent to process the task orchestration detection result, the understanding defects of the plurality of initial subtasks and the initial orchestration logic for the demand intention or the basic conditions can be corrected, and then the demand intention can be processed under the condition of correcting the understanding defects to perform the task orchestration task, obtaining a plurality of target subtasks that meet the video demand conditions and the dependency relationships between the plurality of target subtasks.

[0095] According to an embodiment of the present disclosure, in the case where the current task orchestration detection result indicates that the plurality of intermediate subtasks and intermediate dependency relationships obtained by using the first designated sub-agent to perform a task orchestration subtask based on the demand intention and the task orchestration detection result still do not meet the video demand conditions, the first designated sub-agent can be used to repeatedly perform the task orchestration detection subtask based on the current task orchestration detection result and the demand intention until the current task orchestration detection result indicates that the plurality of generated intermediate subtasks and intermediate dependency relationships meet the video demand conditions, and the currently generated plurality of intermediate subtasks and intermediate dependency relationships are used as the generated plurality of target subtasks and dependency relationships.

[0096] According to an embodiment of the present disclosure, by generating a plurality of initial subtasks and initial dependencies through a first designated sub-agent, and by using a second designated sub-agent to perform task orchestration detection on the plurality of initial subtasks and initial dependencies, it is possible to implement the inspection and generation of task orchestration results in a way that utilizes the collaboration of multiple agents, avoiding the situation where the same agent generates the orchestration logic of multiple target subtasks and self-reflects on the orchestration logic, which makes it difficult to detect the defect types of the orchestration results. Thus, based on the task orchestration detection results generated by the second designated sub-agent, the first designated sub-agent can be instructed to correct errors, achieving accurate correction of the initial subtasks and initial dependencies.

[0097] According to an embodiment of the present disclosure, the task orchestration detection results characterize at least one of the following defect types: there is an execution logic conflict in the initial dependencies; the types of the plurality of initial subtasks among the plurality of initial subtasks do not match the requirement types represented by the requirement intention.

[0098] There is an execution logic conflict in the initial dependencies, which can be understood as the initial task that executes first among the plurality of initial subtasks requires the sub-execution result of the initial subtask that executes later. For example, the copywriting generation task is the initial subtask that executes last, while the speech synthesis subtask is the initial subtask that executes first. However, the speech synthesis subtask requires the target copywriting generated by the copywriting generation task to execute the speech generation subtask. Or, there is an execution logic conflict in the initial dependencies can also be understood as other types of sub-task execution logic defects, such as execution logic defects like the initial subtask execution entering an infinite loop.

[0099] The types of the plurality of initial subtasks among the plurality of initial subtasks do not match the requirement types represented by the requirement intention. For example, among the plurality of initial subtasks, there may be a lack of a graphic element generation subtask that matches the video illustration requirement represented by the requirement intention. However, it is not limited to this. As long as the task orchestration detection results can indicate which requirement types do not match the requirement intention represented.

[0100] Figure 4 Schematically shows the principle diagram of generating task orchestration results using a designated agent according to an embodiment of the present disclosure.

[0101] As Figure 4As shown, the specified agent 410 may include an intention recognition sub-agent 411, a first specified sub-agent 412, and a second specified sub-agent 413. The object requirement information 401 of the target object may be "it is necessary to generate short videos for preventing disease A for public popularization". The intention recognition sub-agent 411 is used to process the object requirement information 401 to obtain the requirement intentions of "short video", "preventing disease A", and "public popularization". The first specified sub-agent 412 is used to process the requirement intentions to perform task orchestration tasks, obtaining a plurality of initial subtasks and initial dependency relationships. The second specified sub-agent 413 performs a task orchestration detection subtask on the plurality of initial subtasks and initial dependency relationships based on the short video conditions represented by the requirement intentions, the video theme requirement condition of "preventing disease A", and the scenario requirement condition of "public popularization", obtaining a task orchestration detection result. When the task orchestration detection result does not meet any of the video requirement conditions, the second specified sub-agent 413 sends the task orchestration detection result to the first specified sub-agent 412 to inform the first specified sub-agent 412 that it is necessary to re-perform the task orchestration task according to the defect type indicated by the task orchestration detection result and the requirement intentions, thereby obtaining the current plurality of intermediate subtasks and intermediate orchestration logics. The second specified sub-agent 413 performs a task orchestration detection check on the current plurality of intermediate subtasks and intermediate orchestration logics based on the video requirement conditions represented by the requirement intentions. When the task orchestration detection result indicates that the video requirement conditions are met, the specified agent 410 distributes the task orchestration result 402 that meets the video requirement conditions to a plurality of target agents. The task orchestration result 402 may include a plurality of target subtasks represented by nodes and the dependency relationships between the plurality of target subtasks represented by directed edges.

[0102] According to an embodiment of the present disclosure, using a first sub-agent of a target agent to determine sub-task execution elements for controlling the execution process of a target sub-task based on sub-task requirement information includes: using the first sub-agent to process the sub-task requirement information and the associated sub-execution results of associated sub-tasks in the target task to obtain sub-task execution elements.

[0103] According to an embodiment of the present disclosure, there is a dependency relationship between an associated sub-task and a target sub-task. For example, the associated sub-task may be a target sub-task that is executed and completed before the execution of the target sub-task in the task orchestration result. The associated sub-execution result may be obtained from an associated target agent that has a dependency relationship with the current target agent.

[0104] According to an embodiment of the present disclosure, using a first sub-agent to process sub-task requirement information and associated sub-execution results of associated sub-tasks in a target task may include using the sub-task requirement information and the associated sub-execution results as prompt information, and using the first sub-agent to receive the input prompt information, so as to prompt the first sub-agent to specifically understand the requirement key points of the current target sub-task according to the associated sub-execution results, improve the description accuracy of the sub-task execution elements for the task attributes of the sub-task, and further enable the second sub-agent to execute the target sub-task based on the relatively accurate sub-task execution elements, thereby improving the accuracy of the subsequent obtained target sub-execution results.

[0105] According to an embodiment of the present disclosure, using a first sub-agent to process sub-task requirement information and associated sub-execution results of associated sub-tasks in a target task to obtain sub-task execution elements includes: using the first sub-agent to process the sub-task requirement information, the associated sub-execution results, and the object requirement information of the target object to obtain sub-task execution elements.

[0106] According to an embodiment of the present disclosure, the object requirement information represents the requirement attributes of the target object for the target video. The target object may be a user object that obtains the target video.

[0107] In one example, the sub-task requirement information may include the full requirement intention of the target object, or may further include the object requirement information, so that the first sub-agent can process based on the full requirement information of the target object for the target task requirement as prompt information, improve the representation accuracy of the sub-task execution elements for the requirement intention, and ensure that the sub-task execution elements can maintain logical coherence and requirement consistency with the sub-task requirement conditions represented by the associated sub-tasks with a dependency relationship, avoid conflicts in the sub-task execution elements, and improve the accuracy and adaptability of the target sub-execution results.

[0108] According to an embodiment of the present disclosure, the target sub-task includes a storyboard data sub-task. Among them, using the first sub-agent to process the sub-task requirement information and the associated sub-execution results of the associated sub-tasks in the target task may include: using the first sub-agent to process the target copywriting and the sub-storyboard task requirement information for the storyboard data sub-task to obtain sub-storyboard task execution elements.

[0109] According to an embodiment of the present disclosure, the target copywriting is determined by executing a copywriting generation sub-task.

[0110] In one example, the target copywriting may be generated by a copywriting generation intelligent agent processing the copywriting generation sub-task requirement information, and the target copywriting meets the copywriting requirement conditions in the copywriting generation sub-task requirement information.

[0111] According to an embodiment of the present disclosure, the split mirror task execution element is used to represent that the display attribute of the target element in the target video is adapted to the target text content of the target copywriting. The target element includes at least one of a graphic element and a subtitle text. The display attribute includes at least one of a display duration and a display timing. For example, the split mirror task execution element may indicate that the split mirror data generated by the second intelligent agent will make the subtitle text, image element, and copywriting voice representing the same content or semantics have the same display timing. Or, for another example, the split mirror task execution element may indicate that the split mirror data generated by the second intelligent agent will make the subtitle text, image element, and copywriting voice representing the same requirement intention or the same semantic content have the same display duration and display timing to maintain the display consistency and coordination of the target video.

[0112] In one example, the first sub-intelligent agent can be utilized to process the split mirror task execution element and related sub-execution results such as the target copywriting, as well as at least one of the object requirement information and the requirement intention, to generate the target split mirror data, so that the target split mirror data can synchronously display the key copywriting text subtitle, the voice data corresponding to the key copywriting text, and the key text explanation graphic element in the target video through the same display attribute for the key copywriting text in the copywriting text that matches the requirement intention, thereby improving the display effect of the target video.

[0113] According to an embodiment of the present disclosure, the second sub-intelligent agent of the target intelligent agent executes the target sub-task based on the sub-task execution element, and obtaining the target sub-execution result may include: the second sub-intelligent agent of the target intelligent agent executes the target sub-task based on the sub-task execution element to obtain the sub-execution result; the third sub-intelligent agent of the target intelligent agent detects the sub-execution result to obtain the sub-task detection result; in the case where the sub-task detection result does not meet the sub-task requirement condition in the sub-task execution element, the second sub-intelligent agent executes the target sub-task based on the sub-task detection result to obtain the target sub-execution result that meets the sub-task requirement condition.

[0114] According to an embodiment of the present disclosure, the third sub-intelligent agent may be an intelligent agent that is different from both the first sub-intelligent agent and the second sub-intelligent agent. The first sub-intelligent agent, the second sub-intelligent agent, and the third sub-intelligent agent in the target intelligent agent may be intelligent agents for performing different reasoning functions. The sub-execution result generated by the second sub-intelligent agent may be the execution result of the target sub-task that has not been detected. By using the third intelligent agent to detect the sub-execution result, in the case where the sub-task detection result does not meet the sub-task requirement condition, the second sub-intelligent agent is controlled to execute the target sub-task under the condition of paying more attention to the defect type according to the defect type indicated by the sub-task detection result, so that the generated target sub-task meets the sub-task requirement condition.

[0115] According to an embodiment of the present disclosure, the sub-task requirement condition can be understood as the condition in the sub-task execution elements that indicates the conditions that the sub-execution result output by the second sub-agent needs to meet. The sub-task requirement condition can be input into the second sub-agent as a prompt message to control the second agent to execute the target sub-task. By using the third sub-agent to detect the sub-execution result based on the sub-task requirement condition, the sub-task detection result can more accurately represent the semantic understanding defect of the second agent for the sub-task requirement condition, and the second sub-agent can be used to execute the target sub-task based on the defect type indicated by the sub-task detection result, so that the second sub-agent can focus on the understanding defect type and execution defect type for the sub-task requirement condition, thereby realizing controlling the second sub-agent to accurately execute the target sub-task based on the sub-task execution elements based on the sub-task detection result as a prompt message, and obtaining a target sub-execution result that meets the sub-task requirement condition.

[0116] In one example, using the second sub-agent to execute the target sub-task based on the sub-task detection result may include using the sub-task detection result and the sub-task execution elements as prompt messages to control the second sub-agent to execute the target sub-task and obtain the current intermediate sub-execution result. By using the third sub-agent to detect the current intermediate sub-execution result based on the sub-task requirement condition, when the sub-task detection result indicates that the current intermediate sub-execution result meets the sub-task requirement condition, the current intermediate sub-execution result is determined as the target sub-execution result. When the sub-task detection result indicates that the current intermediate sub-execution result does not meet the sub-task requirement condition, the second sub-agent can be used to iteratively execute the target sub-task based on the current sub-task detection result until the intermediate sub-execution result meets the sub-task requirement condition.

[0117] According to an embodiment of the present disclosure, by using the semantic understanding capabilities and execution functions of the first sub-agent, the second sub-agent, and the third sub-agent respectively to perform sub-task execution element reasoning, target sub-task execution, and sub-execution result detection, it is possible to perform task execution key point analysis, task precise execution, and execution result detection and feedback for the target sub-task by using different agents respectively, and to finely execute the target sub-task through multi-agent collaboration to avoid situations such as execution hallucinations and semantic understanding errors that occur in the generative model for constructing the same agent, and improve the execution accuracy and execution efficiency of the target sub-task.

[0118] According to an embodiment of the present disclosure, using the third sub-agent of the target agent to detect the sub-execution result and obtain the sub-task detection result includes: using the third sub-agent to detect the sub-execution result based on the sub-task requirement condition and the associated sub-execution result of the associated sub-task in the target task to obtain the sub-task detection result.

[0119] According to an embodiment of the present disclosure, there is a dependency relationship between an associated subtask and a target subtask, and the associated sub-execution result can represent a target sub-execution result that meets the subtask requirement conditions for the associated subtask. Using the second sub-agent to execute the target subtask based on the subtask detection result may include using the second sub-agent to use at least one of the subtask detection result, the associated sub-execution result that meets the subtask requirement conditions for the associated subtask, the object requirement information, and the requirement intention as prompt information to execute the target subtask. This can enable the second sub-agent to fully learn the global information in the target task and the already generated target sub-execution result, thereby improving the second sub-agent's ability to understand global information, making the generated target sub-execution result maintain logical coherence, semantic consistency, and style coordination with the target sub-execution results of other target subtasks, and further improving the display effect of the target video.

[0120] In one example, the first sub-agent, the second sub-agent, and the third sub-agent in the target agent can perform an inference function based on the full amount of information already generated during the execution of the current target task. For example, the second sub-agent can execute the target subtask based on the full amount of information already generated during the execution of the current target task, and the third sub-agent can detect the subtask execution result based on the full amount of information already generated during the execution of the current target task to obtain a subtask detection result. The full amount of information already generated during the execution of the current target task includes object requirement information, requirement intention, subtask execution elements, sub-execution results, subtask execution elements of the associated subtask, sub-execution results that meet the subtask requirement conditions of the associated subtask, etc. This can enable multiple sub-agents to pay attention to the subtask execution elements and target sub-execution results of other target subtasks based on the full amount of information for the target task, so as to keep multiple target subtasks logically coherent and style-consistent, and avoid missing or misunderstanding the requirement attributes represented by the requirement information, improving the display effect of the target video and the requirement matching degree with the target object.

[0121] According to an embodiment of the present disclosure, the target subtask includes a graphic element generation subtask, and the sub-execution result includes graphic elements to be displayed in the target video. The graphic elements are synchronously displayed in the target video with video caption text and voice data representing the same or similar meanings to improve the display effect of the target video for key information.

[0122] According to an embodiment of the present disclosure, using the third sub-agent of the target agent to detect the sub-execution result includes: using the third agent to detect the graphic elements based on at least one of the target copywriting and the target storyboard data, and the graphic element requirement conditions for the graphic element generation subtask to obtain a graphic element detection result.

[0123] According to an embodiment of the present disclosure, at least one of the target copywriting and the target storyboard data is an associated sub-execution result, and the associated sub-execution result is determined by executing an associated sub-task in the target task. The associated sub-task includes a copywriting generation sub-task and / or a storyboard data generation sub-task that has a dependency relationship with the graphic element generation sub-task.

[0124] According to an embodiment of the present disclosure, by using at least one of the target copywriting and the target storyboard data, and the graphic element requirement conditions for the graphic element generation sub-task as prompt information, to control the third intelligent agent to detect the graphic elements, the graphic element detection result can more accurately represent the correlation between the currently generated graphic elements and the already generated target copywriting text and target storyboard data. Thus, the second intelligent agent can be controlled to execute the graphic element generation sub-task according to the defect type indicated by the graphic element detection result, to generate target graphic elements that meet the graphic element requirement conditions. This can make the target graphic elements, the target copywriting, and the target storyboard data meet the same demand intention, avoid the appearance of semantically irrelevant images and texts in the target video, maintain the logical coherence and consistency of the expression of the target video content, and improve the display effect of the target video.

[0125] It should be noted that the target copywriting can be determined by a copywriting generation intelligent agent for executing the copywriting generation sub-task, or the target copywriting can also be obtained based on other methods. The target storyboard data can be determined by a storyboard generation intelligent agent for executing the storyboard data generation sub-task, or the target storyboard data can also be obtained based on other methods.

[0126] Figure 5 Schematically shows the schematic diagram of using the target intelligent agent to execute the target sub-task according to an embodiment of the present disclosure.

[0127] As Figure 5 shown, the target intelligent agent 510 may include a first sub-intelligent agent 511, a second sub-intelligent agent 512, and a third sub-intelligent agent 513. The target intelligent agent 510 is a graphic element generation sub-intelligent agent for generating target graphic elements. Taking the target copywriting 501 as an associated sub-execution result, the first sub-intelligent agent 511 performs task element analysis based on the target copywriting 501 and the object requirement information, and obtains a plurality of sub-task execution elements. The sub-task execution elements may include "exercise graph", "sleep graph", video theme requirement condition "preventing disease A", style requirement condition "popular science for the public", etc. The sub-task execution elements may also include any task parameters, task execution steps, etc. such as the literature type for reference, the name of the authoritative website, etc. that are used to control the execution process of the copywriting generation sub-task.

[0128] The second sub-agent 512 generates sub-tasks for graph elements based on multiple sub-task execution elements, and obtains multiple graph elements as execution results. The third sub-agent 513 is used to detect the multiple graph elements based on the target copywriting 501 and the sub-task requirement conditions, and obtains the sub-task detection results. In the case where the sub-task detection results indicate that the graph element requirement conditions are not met, the third sub-agent 513 feeds back the sub-task detection results to the second sub-agent 512. The second sub-agent 512 generates sub-tasks for graph elements based on the sub-task detection results and the sub-task execution elements, and obtains new graph elements. In this way, the third sub-agent 513 can be used to detect the new graph elements based on the target copywriting 501 and the sub-task requirement conditions until the graph elements currently generated by the second sub-agent 512 meet the graph element sub-task requirement conditions, and the target graph element 502 is obtained.

[0129] According to an embodiment of the present disclosure, the sub-task detection results include graph element detection results, and the graph element detection results are obtained by detecting the graph elements determined by generating sub-tasks for the execution of graph elements.

[0130] According to an embodiment of the present disclosure, the graph element detection results characterize at least one of the following defect types: the image style of the graph element does not match the copywriting style of the target copywriting; the degree of semantic difference between the image content of the graph element and the text semantics of the target copywriting does not meet the semantic similarity conditions; the arrangement of the graph elements does not meet the preset arrangement conditions.

[0131] According to an embodiment of the present disclosure, the image style and the copywriting style can be any preset style attributes such as lively, serious, and second-dimensional. By using sample images or sample texts labeled with style attribute tags to fine-tune the sub-agents respectively, the sub-agents can be made to understand the image style of the graph elements or the copywriting style of the target copywriting. By indicating that the image style of the current graph elements of the second sub-agent does not match the copywriting style, the second sub-agent is controlled to pay more attention to the consistency of the graph element style during the execution of the target sub-task, and the generation efficiency of the target graph elements is improved.

[0132] According to an embodiment of the present disclosure, the image content can be element objects such as text, people, animals, etc. represented by the graph elements. The third sub-agent can obtain the image content through text recognition or target detection algorithms, and compare the image content with the corresponding copywriting text content to determine whether the copywriting text and the image content represent the same or similar element objects, so as to avoid the situation where the displayed text or voice content in the target video is completely irrelevant to the image content.

[0133] The semantic difference degree between the image content of the graphic element and the text semantics of the target copy does not meet the semantic similarity condition. For example, the image content can be a half-body photo of "Zhang San", while the corresponding text content of the target copy is expressed as "Li Si once won the championship". It can be seen that the text difference degree between the image content and the target copy is relatively large. The semantic difference degree between the image content of the graphic element and the text semantics of the target copy does not meet the semantic similarity condition.

[0134] According to an embodiment of the present disclosure, the arrangement manner of the graphic element does not meet the preset arrangement manner condition, which can be understood as that the arrangement shape, arrangement spacing, and arrangement position of multiple sub-elements in the graphic element do not meet the preset arrangement manner condition. By detecting the arrangement manner of the graphic element, the target graphic element can be displayed in the target video according to the requirement intention of the target object, so as to improve the display effect of the target video. At the same time, it can avoid ambiguity caused by the arrangement manner of the graphic element not meeting the preset arrangement manner condition and improve the quality of the target video.

[0135] According to an embodiment of the present disclosure, the sub-task detection result includes the shot data detection result, and the shot data detection result is obtained by detecting the shot data determined by executing the sub-task of generating the shot data.

[0136] According to an embodiment of the present disclosure, the shot data detection result characterizes at least one of the following defect types: the display timing of the graphic element indicated by the shot data does not match the playback timing of the target voice segment; the display duration of the graphic element indicated by the shot data does not match the playback duration of the target voice segment.

[0137] According to an embodiment of the present disclosure, the second sub-agent is instructed to pay attention to the display timing of the graphic element and the playback timing of the target voice segment to be consistent, and the display duration of the graphic element and the playback duration of the target voice segment to be matched through the shot data detection result. Thus, the target voice segment and the target graphic element in the target video can be synchronously displayed through relatively accurate target shot data, so as to improve the consistency and fluency of the target video.

[0138] Figure 6 Schematically shows a schematic diagram of a video generation method based on multi-agent cooperation according to an embodiment of the present disclosure.

[0139] As Figure 6As shown, the target object can send a video generation request based on an intelligent terminal device such as a laptop. The object requirement information 601 is carried in the video generation request, and the object requirement information 601 can record the natural language information "need to generate a short video for preventing disease A for public popularization". The task orchestration agent 611, as the designated agent, performs intent recognition on the object requirement information 601 to obtain the requirement intent. And the first designated sub-agent of the task orchestration agent 611 executes the task orchestration task to obtain multiple initial sub-tasks and initial dependencies. The second designated sub-agent in the task orchestration agent 611 performs a task orchestration detection sub-task on the multiple initial sub-tasks and initial dependencies to obtain a task orchestration detection result. In the case where the task orchestration detection result does not meet the video requirement conditions, the first designated sub-agent of the task orchestration agent 611 loops to execute the task orchestration task based on the task orchestration detection result and the requirement intent or the object requirement information 601 until multiple target sub-tasks that meet the video requirement conditions and the dependencies between the multiple target sub-tasks are obtained. The task orchestration agent 611 selects multiple target sub-tasks that match the requirement intent from the candidate agents in the agent library according to the task types of the multiple target sub-tasks and the ability description information of the multiple candidate agents in the agent library. And the multiple target sub-tasks and the dependencies represented based on the topological structure are sent to the multiple target agents, so that each target agent can know the execution order of the multiple target sub-tasks and the interaction relationship between the target agent and other target agents.

[0140] The multiple target subtasks in the target task for generating the target video may include a copywriting generation subtask, a subtitle generation subtask, a storyboard data generation subtask, a graphic element generation subtask, and a dynamic image generation subtask. The multiple target agents may be a copywriting generation agent 621, a subtitle generation agent 622, a storyboard generation agent 623, a graphic element generation agent 624, and a dynamic image generation agent 625 respectively. The copywriting generation agent 621, the subtitle generation agent 622, the storyboard generation agent 623, the graphic element generation agent 624, and the dynamic image generation agent 625 may each obtain an associated sub-execution result that meets the requirements of the associated subtask from adjacent target agents with dependencies, based on the execution order of the target subtasks indicated by the dependencies, to execute the target subtask and obtain a target sub-execution result. At the same time, the multiple target sub-agents may share their respective subtask execution elements, subtask requirement conditions, target sub-execution results, retrieved database content, and object requirement information 601 input by the target object. This can enable each target agent that executes the target subtask later to obtain the full amount of information that has been generated during the execution of the target task. Thus, the target agent can select the required content from the rights information based on the functions and requirements of the multiple sub-agents, and generate a target sub-execution result based on the obtained content. This can achieve the execution order of the target subtasks represented by the dependencies, continuously update the rights information shared by each target agent using the target sub-execution results, and achieve the automatic transmission of the target sub-execution results to the target agent of the next target subtask to be executed according to the dependencies. Finally, according to the dependencies in the task scheduling result, the target sub-execution results of the multiple target subtasks are summarized, and the multiple target sub-execution results are transmitted to the video template compilation component 631 as video generation materials for video generation. The video materials after summarizing the multiple target sub-execution results may include target copywriting, target subtitle text, target storyboard data, target graphic elements, and target dynamic images.

[0141] The video template compilation component 631 automatically selects the best video generation template according to the demand intention of the target object and the summarized video materials, and renders to obtain the target video 602. By sending the file recording the target video 602 to the intelligent terminal device of the target object, it is convenient for the user to retrieve the target video for viewing or sharing at any time.

[0142] Such as Figure 6As shown, each target agent contains a first sub-agent for performing sub-task execution element reasoning, a second sub-agent for performing the execution function of the target sub-task, and a third sub-agent for performing the sub-execution result detection function. The first sub-agent inside each target agent serves as a task analysis sub-agent, analyzes all information such as the associated sub-execution result or object requirement information transmitted by the upper-level agent, and obtains sub-task requirement elements, so that the sub-task requirement elements can plan and control the execution process of the target sub-task, facilitating the second sub-agent to accurately execute the target sub-task. The third sub-agent is used to detect the sub-execution result output by the second sub-agent, and inform the second sub-agent of the defect type indicated by the sub-task detection result to improve the execution process of the target sub-task, so that an analysis-execution-feedback closed-loop task execution mechanism can be formed by repeatedly executing the target sub-task and performing detection feedback, thereby quickly generating a target sub-execution result that meets the sub-task requirement conditions. In this way, the target video can be quickly generated by quickly generating the target sub-execution result, realizing video industrial production based on the cooperation of multiple target agents and multiple sub-agents, and achieving video production scale and customization.

[0143] The sub-agents involved in this embodiment, including but not limited to the first specified sub-agent, the first sub-agent, etc., can serve as the basic units of the agent. The sub-agent is built based on large models such as the large language model or the visual large model, and can simultaneously execute the reasoning function of the agent by interacting with external tools such as search engines, material libraries, and code compilers, and can efficiently complete tasks such as video script writing, creative copywriting generation, and multi-modal video material production.

[0144] The target agent and the specified agent involved in this embodiment can enable multiple sub-agents to cooperate to complete relatively complex target sub-tasks or task choreography tasks based on the respective execution functions of the multiple sub-agents. The multiple sub-agents in the target agent can realize the analysis-execution-feedback closed-loop function for the target sub-task or task choreography by respectively assuming the functional roles of task analysis, task execution, and result feedback, realize continuous optimization and adjustment of the target sub-task execution result based on the defect type of the feedback, and realize the fast and efficient execution of the target sub-task.

[0145] According to the embodiments of the present disclosure, a general agent can be used to detect the sub-execution result of each target sub-task, which is suitable for performing general detection tasks. However, it is not limited thereto. Multiple sub-expert agents can also be set for the main agent, and the sub-expert agents are suitable for performing personalized detection tasks.

[0146] The model structure configured in the general intelligent agent or sub-expert intelligent agent is not limited. For example, it may include a deep learning model with one or more of the attention mechanism, convolutional network, recurrent network, and long short-term memory network. However, it is not limited to this, and it may also be a large language model with more powerful functions. As long as it is a model that can be used to perform the detection task. The difference is that by using multiple sub-expert intelligent agents, a mixture of experts mechanism can be formed, combining multiple targeted and personalized sub-expert intelligent agents, and selecting different sub-expert intelligent agents to perform the detection task for detection tasks with different sub-task requirements.

[0147] Figure 7A Schematically shows an effect diagram of the target video according to an embodiment of the present disclosure.

[0148] As Figure 7A shown, the target video 700 can play the target voice representing the target copy by driving the lip movements of the virtual object 701. During the playback of the target voice, the subtitle text "Don't eat sugar, don't drink sugary drinks" representing the target voice will be displayed in the video frame of the target video 700, and the subtitle text can be highlighted based on underlining and bolding. The target graphic element 702 can be the graphic content representing the text content of the subtitle text "Don't eat sugar, don't drink sugary drinks". The subtitle text "Don't eat sugar, don't drink sugary drinks", the speech data segment, and the target graphic element 702 representing the same meaning will be synchronously displayed in the video frame of the target video based on the same display timing and display duration indicated by the target storyboard data.

[0149] Figure 7B Schematically shows an effect diagram of the health science popularization video according to an embodiment of the present disclosure.

[0150] As Figure 7BAs shown, the target video can be a health science popularization video T710. In the health science popularization video T710, the virtual image can be driven by the target voice data corresponding to the target copywriting to explain knowledge related to taking medicine. The first image element T711 and the second image element T712 in the health science popularization video T710 can be obtained by the graph element generation agent executing the graph element generation subtask. The first image element T711 and the second image element T712 meet the theme requirement conditions of "knowledge explanation of medicine taking" for user needs. The subtitle text "Recall whether you have eaten anything special within 48 hours..." in the health science popularization video T710 is the target sub-execution result obtained by the subtitle generation agent executing the subtitle generation subtask. In the health science popularization video T710, the first image element T711 and the subtitle text "Recall whether you have eaten anything special within 48 hours..." can be displayed based on the appropriate display timing and display duration. At the same time, the key information "48 hours" in the subtitle text can be enlarged in font and modified in color, so that the health science popularization video T710 can enable the audience to more intuitively pay attention to the dragon fruit and medicine displayed in the first image element T711 during taking medicine or treatment, improve the display effect of the health science popularization video, and improve the matching degree between the target video and user needs.

[0151] It should be noted that Figure 7B The black box in the health science popularization video T710 shown is to protect relevant privacy information or important information, and it is not a display defect of the target video.

[0152] Figure 7C Schematically shows the effect diagram of the legal popularization video according to the embodiment of the present disclosure.

[0153] As Figure 7CAs shown, the target video can be the legal popularization video T720. In the legal popularization video T720, the virtual image can be driven by the target voice data corresponding to the target copywriting to explain the relevant legal knowledge about the prohibition of carrying controlled knives. The third image element T721 and the fourth image element T722 in the legal popularization video T720 can be obtained by the figure element generation intelligent agent executing the figure element generation subtask. The third image element T721 and the fourth image element T722 meet the theme requirement conditions of "explanation of controlled knives" for user needs. The subtitle text "Because of their potential danger..." in the legal popularization video T720 is the target sub-execution result obtained by the subtitle generation intelligent agent executing the subtitle generation subtask. In the legal popularization video T720, the third image element T721 and the subtitle text "Because of their potential danger..." can be displayed based on the appropriate display timing and display duration, so that the legal popularization video T720 can enable the audience to more intuitively and timely pay attention to various controlled knives in the third image element T721 during the explanation process of the virtual image, strengthen the understanding of the relevant legal knowledge, improve the display effect of the legal popularization video, and improve the matching degree between the target video and user needs.

[0154] It should be noted that Figure 7C The black box in the legal popularization video T720 shown is to protect relevant privacy information or important information, and it is not a display defect of the target video.

[0155] It should be noted that the acquisition, processing, and display of data involved in any embodiment of the present disclosure are all implemented under the condition of obtaining the authorization of relevant users or institutions, and necessary encryption or desensitization measures are adopted during the implementation of the embodiments of the present disclosure to avoid information leakage.

[0156] Figure 8 Schematically shows a block diagram of a video generation device based on multi-agent collaboration according to an embodiment of the present disclosure.

[0157] As Figure 8 shown, the video generation device 800 based on multi-agent collaboration includes: an execution module 810 and a target video generation module 820.

[0158] The execution module 810 is used to utilize at least one target intelligent agent to execute the target subtask in the target task to obtain the target sub-execution result.

[0159] A target video generation module 820 is configured to generate a target video based on a target sub - execution result. The target agent performs the following operations: obtaining sub - task requirement information for a target sub - task; using a first sub - agent of the target agent to determine sub - task execution elements for controlling the execution process of the target sub - task based on the sub - task requirement information; using a second sub - agent of the target agent to execute the target sub - task based on the sub - task execution elements to obtain a target sub - execution result.

[0160] According to an embodiment of the present disclosure, the target agent is configured to perform the following operations to use a second sub - agent of the target agent to execute the target sub - task based on the sub - task execution elements to obtain a target sub - execution result: using the second sub - agent of the target agent to execute the target sub - task based on the sub - task execution elements to obtain a sub - execution result; using a third sub - agent of the target agent to detect the sub - execution result to obtain a sub - task detection result; in the case where the sub - task detection result does not meet the sub - task requirement condition in the sub - task execution elements, using the second sub - agent to execute the target sub - task based on the sub - task detection result to obtain a target sub - execution result that meets the sub - task requirement condition.

[0161] According to an embodiment of the present disclosure, the target agent is configured to perform the following operations to use a third sub - agent of the target agent to detect the sub - execution result to obtain a sub - task detection result: using the third sub - agent to detect the sub - execution result based on the sub - task requirement condition and the associated sub - execution result of an associated sub - task in the target task to obtain a sub - task detection result, where there is a dependency relationship between the associated sub - task and the target sub - task.

[0162] According to an embodiment of the present disclosure, the target sub - task includes a graph element generation sub - task, and the sub - execution result includes graph elements for display in the target video.

[0163] According to an embodiment of the present disclosure, the target agent is configured to perform the following operations to use a third sub - agent of the target agent to detect the sub - execution result: using the third agent to detect the graph elements based on at least one of the target copywriting and the target storyboard data, and the graph element requirement condition for the graph element generation sub - task to obtain a graph element detection result, where at least one of the target copywriting and the target storyboard data is an associated sub - execution result, and the associated sub - execution result is determined by executing an associated sub - task in the target task, and the associated sub - task includes a copywriting generation sub - task and / or a storyboard data generation sub - task that has a dependency relationship with the graph element generation sub - task.

[0164] According to an embodiment of the present disclosure, the sub - task detection result includes a graph element detection result, and the graph element detection result is obtained by detecting the graph elements determined by executing the graph element generation sub - task.

[0165] According to an embodiment of the present disclosure, the figure element detection result characterizes at least one of the following defect types: the image style of the figure element does not match the copy style of the target copy; the degree of difference between the image content of the figure element and the text semantics of the target copy does not meet the semantic similarity condition; the arrangement manner of the figure element does not meet the preset arrangement manner condition.

[0166] According to an embodiment of the present disclosure, the subtask detection result includes the storyboard data detection result, and the storyboard data detection result is obtained by detecting the storyboard data determined by executing the storyboard data generation subtask; wherein, the storyboard data detection result characterizes at least one of the following defect types: the display timing of the figure element indicated by the storyboard data does not match the playback timing of the target voice segment; the display duration of the figure element indicated by the storyboard data does not match the playback duration of the target voice segment.

[0167] According to an embodiment of the present disclosure, the target intelligent agent is configured to perform the following operations to use the first sub-intelligent agent of the target intelligent agent to determine the subtask execution elements for controlling the execution process of the target subtask based on the subtask requirement information: use the first sub-intelligent agent to process the subtask requirement information and the associated sub-execution results of the associated subtasks in the target task to obtain the subtask execution elements, wherein there is a dependency relationship between the associated subtasks and the target subtask.

[0168] According to an embodiment of the present disclosure, the target intelligent agent is configured to perform the following operations to use the first sub-intelligent agent to process the subtask requirement information and the associated sub-execution results of the associated subtasks in the target task to obtain the subtask execution elements: use the first sub-intelligent agent to process the subtask requirement information, the associated sub-execution results, and the object requirement information of the target object to obtain the subtask execution elements, wherein the object requirement information characterizes the requirement attributes of the target object for the target video.

[0169] According to an embodiment of the present disclosure, the target subtask includes a storyboard data subtask.

[0170] According to an embodiment of the present disclosure, the target intelligent agent is configured to perform the following operations to use the first sub-intelligent agent to process the subtask requirement information and the associated sub-execution results of the associated subtasks in the target task:

[0171] Use the first sub-intelligent agent to process the target copy and the sub-storyboard task requirement information for the storyboard data subtask to obtain the sub-storyboard task execution elements, wherein the target copy is determined by executing the copy generation subtask; wherein, the sub-storyboard task execution elements are used to represent that the display attributes of the target elements in the target video are adapted to the target text content of the target copy, and the target elements include at least one of figure elements and caption texts, and the display attributes include at least one of display duration and display timing.

[0172] According to an embodiment of the present disclosure, the subtask execution element includes at least one of the following subtask requirement conditions: a style consistency condition, which characterizes that the expression style of the target subtask execution result matches the requirement style characterized by the object requirement information, where the expression style includes at least one of a text expression style, a voice expression style, and a video frame expression style; a logical coherence condition, which characterizes that there is logical coherence among multiple target subtask execution results of the target task.

[0173] According to an embodiment of the present disclosure, the target task includes multiple target subtasks.

[0174] The video generation device 800 based on multi-agent collaboration is further configured to use a specified agent to perform the following operations: perform intention recognition on the object requirement information of the target object to obtain a requirement intention; determine multiple target subtasks and the dependency relationship between the multiple target subtasks based on the requirement intention; and send the multiple target subtasks and the dependency relationship between the multiple target subtasks to multiple target agents.

[0175] According to an embodiment of the present disclosure, determining multiple target subtasks and the dependency relationship between the multiple target subtasks based on the requirement intention includes: using a first specified sub-agent of the specified agent to perform a task orchestration task based on the requirement intention to obtain multiple initial subtasks and an initial dependency relationship; using a second specified sub-agent of the specified agent to perform a task orchestration detection subtask on the multiple initial subtasks and the initial dependency relationship based on the video requirement conditions represented by the requirement intention to obtain a task orchestration detection result; in the case where the task orchestration detection result does not meet the video requirement conditions, using the first specified sub-agent to perform a task orchestration subtask based on the requirement intention and the task orchestration detection result to obtain multiple target subtasks that meet the video requirement conditions and the dependency relationship between the multiple target subtasks.

[0176] According to an embodiment of the present disclosure, the task orchestration detection result characterizes at least one of the following defect types: there is an execution logic conflict in the initial dependency relationship; the multiple initial subtask types of the multiple initial subtasks do not match the requirement types characterized by the requirement intention.

[0177] According to an embodiment of the present disclosure, the subtask requirement information for the target subtask is determined by a specified agent based on the requirement intention.

[0178] According to an embodiment of the present disclosure, obtaining the subtask requirement information for the target subtask includes: using a target agent to perform requirement information detection on the object requirement information of the target object based on the subtask type of the target subtask to obtain the subtask requirement information.

[0179] According to an embodiment of the present disclosure, the target subtasks include at least one of the following: copywriting generation subtask, storyboard data generation subtask, graphic element generation subtask, and subtitle generation subtask.

[0180] Figure 9 Schematically shows a structural block diagram of an agent of an artificial intelligence according to an embodiment of the present disclosure.

[0181] In an embodiment of the present disclosure, as Figure 9 shown, the AI agent 900 may include an input module 910, a processing module 920, and an output module 930.

[0182] The input module 910 is configured to receive input information;

[0183] The processing module 920 is configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by calling the large model to execute the video generation method based on multi-agent collaboration provided according to an embodiment of the present disclosure;

[0184] The output module 930 is configured to output the output information obtained by the processing module.

[0185] According to an embodiment of the present disclosure, the input module 910 is responsible for receiving or perceiving information such as queries, requests, instructions, signals, or data from the outside world (such as users or the external environment), and converting it into a format that the AI agent 900 can understand and process. The input module 910 is the primary link for the AI agent 900 to interact with the outside world, enabling the AI agent 900 to efficiently and accurately obtain necessary "sensory" information from the outside world and respond to this information.

[0186] In an example, the input module 910 may input the object demand information, subtask demand information, etc. described above.

[0187] In an example, the processing module 920 is the core support for the AI agent 900's ability to handle complex tasks. The processing module 920 can execute the video generation method based on multi-agent collaboration described above.

[0188] In an example, the performance of the processing module 920 may be closely related to the large model on which the AI agent 900 is based. To fully utilize the capabilities of the large model, the internal structure of the processing module 920 can be designed to be highly configurable and extensible to handle various different types of tasks and requirements in real-world scenarios.

[0189] In an example, after the AI agent 900 obtains the demand voice, the processing module 920 can use the target agent constructed based on the large model to process the object demand information to obtain a target video, and transfer the target video to the output module 930.

[0190] It can be understood that although large language models have excellent language understanding and generation capabilities, like humans, they can only solve a limited number of tasks without the aid of any tools. When the AI agent 900 is given the ability to call tools, it can perform tasks such as completing mathematical operations with the help of a calculator, performing data analysis with the help of Python, and obtaining weather forecasts with the help of a search engine.

[0191] In the example, the output module 930 can output the target video described above.

[0192] The AI agent 900 according to the embodiments of the present disclosure can simply and effectively improve the degree of intelligence, and improve flexibility and versatility.

[0193] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0194] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0195] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method described above.

[0196] According to an embodiment of the present disclosure, a computer program product includes a computer program, and the computer program implements the method described above when executed by a processor.

[0197] Figure 10 A schematic block diagram of an example electronic device that can be used to implement the multi-agent collaboration-based video generation method according to the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0198] As Figure 10As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0199] Multiple components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, an optical disc, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0200] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 executes the various methods and processes described above, such as a video generation method based on multi-agent collaboration. For example, in some embodiments, the video generation method based on multi-agent collaboration can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the video generation method based on multi-agent collaboration described above can be executed. Alternatively, in other embodiments, the computing unit 1001 can be configured to execute the video generation method based on multi-agent collaboration by any other appropriate means (e.g., by means of firmware).

[0201] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0202] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0203] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0204] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0205] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0206] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating a blockchain.

[0207] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0208] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A video generation method based on multi-agent collaboration, comprising: Using at least one target intelligent agent to execute a target subtask in the target task to obtain a target subtask execution result; as well as A target video is generated based on the target sub-execution result, wherein the target agent performs the following operations: Acquiring subtask requirement information for the target subtask; Determine, using the first sub-agent of the target agent based on the sub-task requirement information, a sub-task execution element for controlling the execution process of the target sub-task; The second sub-agent of the target agent is used to execute the target sub-task based on the sub-task execution element to obtain the target sub-execution result.

2. The method according to claim 1, wherein: The second sub-agent of the target agent is used to execute the target sub-task based on the sub-task execution element to obtain the target sub-task execution result, which includes: Utilizing the second sub-agent of the target agent to execute the target sub-task based on the sub-task execution element to obtain a sub-execution result; Using the third sub-agent of the target agent to detect the sub-execution result to obtain a sub-task detection result; When the subtask detection result does not satisfy the subtask requirement condition in the subtask execution element, the second sub-agent is used to execute the target subtask based on the subtask detection result to obtain a target subtask execution result that satisfies the subtask requirement condition.

3. The method according to claim 2, wherein: The using the third sub-agent of the target agent to detect the sub-execution result to obtain the sub-task detection result includes: The third sub-agent is used to detect the sub-execution result based on the sub-task requirement condition and the associated sub-execution result of the associated sub-task in the target task to obtain the sub-task detection result, wherein the associated sub-task has a dependency relationship with the target sub-task.

4. The method according to claim 2, wherein: The target subtask includes a graphic element generation subtask, and the sub-execution result includes a graphic element for displaying in the target video; The step of using the third sub-agent of the target agent to detect the sub-execution result includes: The third intelligent agent is used to detect the graphic elements based on at least one of the target text and the target storyboard data, and the graphic element requirement conditions for the graphic element generation subtask, to obtain a graphic element detection result, wherein at least one of the target text and the target storyboard data is an associated sub-execution result, and the associated sub-execution result is determined by executing an associated subtask in the target task, and the associated subtask includes a text generation subtask and / or a storyboard data generation subtask that has a dependency relationship with the graphic element generation subtask.

5. The method according to claim 2, wherein: The subtask detection result includes a graph element detection result, and the graph element detection result is obtained by detecting the graph elements determined by executing the graph element generation subtask; The image element detection result represents at least one of the following defect types: The image style of the graphic element does not match the text style of the target text; The semantic difference between the image content of the graphic element and the text of the target text does not satisfy the semantic similarity condition; The arrangement of the graphic elements does not satisfy the preset arrangement conditions.

6. The method according to claim 2, wherein: The subtask detection result includes a storyboard data detection result, and the storyboard data detection result is obtained by detecting the storyboard data determined by executing the storyboard data generation subtask; The detection result of the split-mirror data represents at least one of the following defect types: The display timing of the graphic elements indicated by the storyboard data does not match the playback timing of the target voice segment; The display duration of the graphic elements indicated by the storyboard data does not match the playback duration of the target voice segment.

7. The method according to claim 1, wherein: The first sub-agent of the target agent determines the sub-task execution element for controlling the execution process of the target sub-task based on the sub-task requirement information, including: The subtask requirement information and the associated sub-execution results of the associated subtasks in the target task are processed by the first sub-agent to obtain the subtask execution elements, wherein the associated subtasks have a dependency relationship with the target subtask.

8. The method according to claim 7, wherein: The step of using the first sub-agent to process the sub-task requirement information and the associated sub-execution results of the associated sub-tasks in the target task to obtain the sub-task execution elements includes: The subtask execution element is obtained by using the first sub-agent to process the subtask requirement information, the associated sub-execution result and the object requirement information of the target object, wherein the object requirement information represents the requirement attribute of the target object for the target video.

9. The method according to claim 7, wherein: The target subtask includes a storyboard data subtask; The step of using the first sub-agent to process the sub-task requirement information and the associated sub-execution results of the associated sub-tasks in the target task includes: Using the first sub-agent to process the target text and the storyboard task requirement information for the storyboard data subtask, to obtain the storyboard task execution element, wherein the target text is determined by executing the text generation subtask; Among them, the split mirror task execution element is used to indicate that the display properties of the target element in the target video are adapted to the target text content of the target copy, the target element includes at least one of a graphic element and a subtitle text, and the display properties include at least one of a display duration and a display timing.

10. The method according to any one of claims 1, 2 or 7, wherein: The subtask execution element includes at least one of the following subtask requirement conditions: A style consistency condition, characterizing the expression style of the target sub-execution result, which matches the demand style characterizing the object demand information, wherein the expression style includes at least one of a text expression style, a voice expression style, and a video screen expression style; The logical coherence condition indicates that there is logical coherence between the multiple target sub-execution results of the target task.

11. The method according to claim 1, wherein: The target task includes a plurality of target subtasks; the method further includes using a designated agent to perform the following operations: Performing intention recognition on the object demand information of the target object to obtain the demand intention; Determining a plurality of target subtasks and dependencies between the plurality of target subtasks based on the demand intention; as well as Sending multiple target subtasks to multiple target agents, as well as dependencies between the multiple target subtasks.

12. The method according to claim 11, wherein: The determining of the plurality of target subtasks based on the demand intention and the dependency relationship between the plurality of target subtasks includes: Utilizing the first designated sub-agent of the designated agent to perform the task scheduling task based on the demand intention, and obtaining a plurality of initial sub-tasks and initial dependency relationships; Using a second designated sub-agent of the designated agent to perform a task scheduling detection subtask on the plurality of initial subtasks and initial dependency relationships based on the video demand condition represented by the demand intent, to obtain a task scheduling detection result; When the task scheduling detection result does not meet the video requirement condition, the first designated sub-agent is used to perform the task scheduling subtask based on the requirement intention and the task scheduling detection result to obtain multiple target subtasks that meet the video requirement condition, as well as the dependency relationships between the multiple target subtasks.

13. The method according to claim 11, wherein: The task scheduling detection result represents at least one of the following defect types: The initial dependency relationship has an execution logic conflict; The multiple initial subtask types of the multiple initial subtasks do not match the requirement type represented by the requirement intention.

14. The method according to claim 11, wherein: The subtask requirement information for the target subtask is determined using the designated agent based on the requirement intention.

15. The method according to claim 1, wherein: The obtaining of subtask requirement information for the target subtask includes: The target agent is used to perform demand information detection on the object demand information of the target object based on the subtask type of the target subtask to obtain the subtask demand information.

16. The method according to claim 1, wherein: The target subtask includes at least one of the following: Copywriting generation subtask, storyboard data generation subtask, graphic element generation subtask, and subtitle generation subtask.

17. A video generation device based on multi-agent collaboration, comprising: An execution module, used to use at least one target agent to execute a target subtask in a target task and obtain a target sub-execution result; as well as A target video generation module is used to generate a target video based on the target sub-execution result, wherein the target agent performs the following operations: Acquiring subtask requirement information for the target subtask; Determine, using the first sub-agent of the target agent based on the sub-task requirement information, a sub-task execution element for controlling the execution process of the target sub-task; The second sub-agent of the target agent is used to execute the target sub-task based on the sub-task execution element to obtain the target sub-execution result.

18. An artificial intelligence agent configured to execute the method as claimed in any one of claims 1 to 16.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 16.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 16.

21. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 16.

Images