A method and system for creating multimedia content based on generative artificial intelligence

Create workflows through interactive interactions in graphical interfaces and customize orchestration functional nodes, solving the problem of single functions of multimedia content creation tools, realizing the flexible combination of generative artificial intelligence and professional software, lowering the threshold for use and improving creative efficiency.

CN118660211BActive Publication Date: 2024-11-15CHENGDU SOBEY DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411140439.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-15
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

The existing multimedia content creation tools have single functions and are difficult to flexibly combine intelligent capabilities, which raises the threshold for use of end users and limits the application of generative artificial intelligence in the field of multimedia content creation.

Method used

Create workflows through interactive graphical interfaces, users can customize orchestration functional nodes to achieve a flexible combination of generative artificial intelligence and professional software, support standardized interface import AI capabilities, and realize multimedia content creation.

Benefits of technology

It lowers the threshold for content creators to use intelligence capabilities, improves the flexibility and efficiency of multimedia content creation, supports low-cost function expansion, and realizes the flexible combination of generative artificial intelligence and professional software.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118660211B_ABST
    Figure CN118660211B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for multimedia content creation of generative artificial intelligence, including: adding the required production nodes and input and output nodes to the workflow canvas by clicking or dragging in the workflow arrangement interface; arranging the connections of each node according to the required multimedia content creation process to create a multimedia content creation workflow; after clicking the production node, a work form operation interface of the production node pops up, and according to the functions of the production node, the corresponding functional requirement prompts are supplemented in the work form, the corresponding processing functions are executed and the results are output; after executing all nodes in the workflow, the final result is output through the output component node of the last stage of the workflow. The present invention realizes the flexible combination of generative artificial intelligence and professional software, and can flexibly customize the production of multimedia content. It supports the import of AI capability nodes provided by different developers through standardized interfaces to achieve low-threshold and low-cost functional expansion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for creating multimedia content using generative artificial intelligence. Background Art

[0002] In traditional multimedia content creation, the production of multimedia content such as audio, video, pictures, and documents usually requires professional tools in their respective fields. For example, Adobe has launched professional audio, image, and video processing software such as Audition, Premiere Pro, Photoshop, and AfterEffects. These tools have built-in capabilities and workflows for specific creative scenarios when they are first released. End users can use these tools to complete the creation of multimedia content for specific creative scenarios. At the same time, the creative scenarios and processes of these tools are single, and the function expansion and iteration are slow, and the access to third-party production capabilities is also relatively weak.

[0003] With the emergence of generative artificial intelligence technology, more and more intelligent capabilities are being applied in the field of multimedia content creation, but most of these intelligent capabilities are provided in the form of single capability nodes and most of them process single objects. End users need to complete switching, input and output between multiple intelligent capabilities in order to integrate multiple intelligent capabilities to complete the creation scenes and results that meet the user's intentions. On the one hand, the manual switching and integration of intelligent capabilities raises the threshold for end users to use intelligent capabilities, consumes creation time, and limits the creation space; on the other hand, it also greatly limits the application of generative artificial intelligence in the field of multimedia content creation.

[0004] Therefore, in order to promote the widespread application of generative artificial intelligence technology in multimedia content creation, lower the technical threshold for content creators to use various intelligent capabilities, and meet the diversified needs of multimedia content creation, it is urgent to combine the mature technology of traditional video production with advanced generative AI capabilities to form a creative method in which end users can flexibly define workflows and conveniently combine various intelligent capabilities, and transform the multimedia content creation method from specific tools that bind linear workflows according to specific scenarios to a system that can integrate various intelligent capabilities and flexibly define workflows. Summary of the invention

[0005] In response to the problems existing in the prior art, a method and system for multimedia content creation based on generative artificial intelligence is provided. Workflow creation is completed by adopting a graphical interface interaction method. Users can arrange functional nodes in a customized manner to create workflows, thereby realizing a flexible combination of generative artificial intelligence and professional software, thereby flexibly producing multimedia content.

[0006] A first aspect of the present invention provides a method for creating multimedia content using generative artificial intelligence, comprising:

[0007] In the workflow arrangement interface, click or drag to add the required production nodes and input and output nodes to the workflow canvas; wherein, the workflow arrangement interface provides a workflow canvas and the input and output nodes and production nodes required for creation. The production nodes are oriented to AI function providers, support standardized and interfaced import, and are used to provide various types of multimedia content creation functions based on generative artificial intelligence and professional media editing software;

[0008] Arrange connections between nodes according to the required multimedia content creation process to create a multimedia content creation workflow;

[0009] After clicking a production node, the work form operation interface of the production node pops up, and the corresponding functional requirement prompts are added in the work form according to the functions of the production node, and the corresponding processing functions are executed and the results are output;

[0010] After executing all nodes in the workflow, the final result is output through the output component node at the last level of the workflow.

[0011] As a preferred solution, when you are not satisfied with the output result of a production node, you can click on the production node and edit the work form for the second time or replace the production node to re-create it; when the production node needs to be replaced or edited for the second time, the output result of the preceding node of the production node remains unchanged, and the subsequent node re-executes the processing process.

[0012] As an optimal solution, during the execution of all nodes of the workflow, if the output quantity of the preceding production node is consistent with the input quantity of the succeeding production node, the computing tasks of the two production nodes are executed in a pipeline form on the same cloud node. When the computing task completion rate of the preceding production node reaches the set value, the image is pulled on the cloud node to build the succeeding production node. After the preceding production node completes the calculation, the output result is saved in the database, and the succeeding production node is notified to start the computing task, and the intermediate results in the memory are shared between tasks; if the output quantity of the preceding production node is inconsistent with the input quantity of the succeeding production node, the preceding production node and the succeeding production node are executed on different cloud nodes. After the preceding production node completes the calculation, the output result is saved in the database, and the succeeding production node is notified to read it. When all the output files of the preceding production nodes are read, the succeeding production node executes the corresponding computing task; if there is no dependency between the two production nodes, the computing tasks are executed in parallel on different cloud nodes.

[0013] A second aspect of the present invention provides a multimedia content creation system of generative artificial intelligence, comprising:

[0014] Node units, including input and output nodes and production nodes, for selection by the workflow arrangement unit; wherein the input and output nodes include video input nodes, image input nodes, audio input nodes, text input nodes, content library input nodes and output component nodes; production nodes are used to provide various types of multimedia content creation functions based on generative artificial intelligence and professional media editing software;

[0015] The workflow arrangement unit is used to select the input and output nodes and production nodes in the node unit, and arrange the selected nodes in the workflow canvas according to the creation needs to establish a workflow. By executing the tasks of each node in the workflow, multimedia content creation can be completed.

[0016] As a preferred solution, each production node includes a work form operation interface for inputting functional requirement prompts to assist the production node in performing corresponding processing work to generate the required multimedia content. After the work of each production node is completed, the node output result can be displayed by clicking the production node.

[0017] As a preferred solution, when a production node in the workflow needs to be replaced or edited again, the output result of the predecessor node of the production node remains unchanged, and the subsequent node re-executes the processing process; wherein, the secondary editing means that the output result of the production node does not meet the requirements, and the work form needs to be edited again and the production node processing process is re-executed.

[0018] As a preferred solution, the node unit is oriented to AI function providers and supports standardized and interfaced import of production nodes; the node unit specifically includes a data input interface, a data output interface, a message notification interface and a parameter modification interface; wherein the data input interface includes a data source input address, a data input type, and a data source input quantity, and the data input interface also reads data from the database according to the data source input address; the data output address includes a data source output address and a data output type; the message notification interface includes receiving messages and sending messages; the parameter modification interface supports a variety of input parameters, including content themes, styles, prompt words, output resolution, etc.

[0019] As a preferred solution, in the workflow orchestration unit, if the output quantity of the preceding production node is consistent with the input quantity of the succeeding production node, the computing task is executed in a pipeline form on the same cloud node. When the computing task completion rate of the preceding production node reaches a set value, the mirror is pulled on the cloud node to build the succeeding production node. After the preceding production node completes the calculation, the output result is saved in the database, and the succeeding production node is notified to start the computing task. The intermediate results in the memory are shared between tasks.

[0020] As a preferred solution, in the workflow orchestration unit, if the output quantity of the preceding production node is inconsistent with the input quantity of the succeeding production node, the preceding production node and the succeeding production node are executed on different cloud nodes. After the preceding production node completes the calculation, the output result is saved in the database and the succeeding production node is notified to read it. Only after all the output files of the preceding production nodes are read, the succeeding production node executes the corresponding calculation task.

[0021] As a preferred solution, in the workflow orchestration unit, if there is no dependency between two production nodes, computing tasks can be executed in parallel on different cloud nodes.

[0022] Compared with the prior art, the beneficial effects of adopting the above technical solution are as follows: the present invention proposes to arrange generative artificial intelligence capabilities and professional software in an interactive way using a workflow arrangement interface. Compared with the previous situation where content creators need to switch multiple websites, multiple systems, adapt to different interaction scenarios, and frequently import and export, it takes longer to achieve the creation of multimedia content works of normal duration and content that meets the intention, it truly realizes the flexible combination of generative artificial intelligence and professional software, and can flexibly customize the production of multimedia content. In addition, it supports the import of AI capability nodes provided by different developers through standardized interfaces to achieve low-threshold and low-cost functional expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of the multimedia content creation method of generative artificial intelligence proposed in the present invention.

[0024] Figure 2 Schematic diagram of the multimedia content creation system of generative artificial intelligence proposed in the present invention.

[0025] Figure 3 Schematic diagram of workflow establishment and execution in one embodiment of the present invention. DETAILED DESCRIPTION

[0026] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as limitations on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0027] Example 1

[0028] This embodiment proposes a method for creating multimedia content using generative artificial intelligence. Figure 1 ,include:

[0029] Step 1. In the workflow arrangement interface, click or drag to add the required production nodes and input and output nodes to the workflow canvas.

[0030] In this embodiment, the workflow arrangement interface provides input and output nodes and production nodes. Among them, the input and output nodes include input nodes and output component nodes, which are mainly used to input content creation materials and output creation results. In one embodiment, the input nodes include video input nodes, image input nodes, audio input nodes, text input nodes, content library input nodes, and output component nodes, etc., which respectively realize the import of video, image, audio, text, and content library. The output component node is mainly used to output and save the output results of the previous node.

[0031] Furthermore, the production node is oriented to AI function providers, and supports standardized and interfaced import of production nodes, which are mainly used to provide various types of multimedia content creation functions based on generative artificial intelligence, as well as professional media editing software or other applications. In other words, the production node can realize the generation and processing of multimedia materials based on generative artificial intelligence.

[0032] In this embodiment, a unified management standard is adopted to realize the unified call management of production nodes, including interfaces such as data input, data output, data reading, message notification, and parameter modification. Among them, the data input interface includes the data source input address, data input type, and data source input quantity. The data input interface can also read data from the database according to the data source input address. The data output interface includes the data source output address and data output type. The message notification interface includes two types: receiving messages and sending messages. The parameter modification interface supports multiple input parameters, such as content theme, style, prompt word, output resolution, etc. Furthermore, the node unit also has global variable fields such as application name, globally unique identifier, data source input quantity, data input type, and data output type to realize node management.

[0033] In actual applications, AI algorithm registrations that conform to standard interfaces can be added to the workflow orchestration unit to form production nodes, and the corresponding generative artificial intelligence capabilities can be realized by calling the production nodes. In one embodiment, the CPU, memory, GPU, video memory and other computing resources required for each production node can be configured according to demand. For multimedia content creators, according to the functions required in the multimedia content creation process, the provided production nodes or input and output nodes in the workflow orchestration interface can be selected and added to the workflow canvas for subsequent execution of corresponding processing.

[0034] Step 2: Arrange the connections of each node according to the required multimedia content creation process to create a multimedia content creation workflow.

[0035] After completing the node selection, you can connect and arrange each node according to the required multimedia content creation process to create a workflow. When arranging each node, generally speaking, the output of the previous node will be used as the input of the next node. In actual applications, there can be multiple inputs and outputs according to its functional needs, which can be divided into single-input single-output nodes, single-input multiple-output nodes, multiple-input single-output nodes, and multiple-input multiple-output nodes.

[0036] Step 3: Click the production node to enter the work form operation interface of the production node, and add corresponding function requirement prompts in the work form according to the functions of the production node, execute the corresponding processing functions and output the results.

[0037] For each production node in the workflow, clicking the production node will pop up the corresponding work form. Multimedia content creators can add corresponding functional requirement prompts in the work form to further control the quality of multimedia content creation. Click the Save button to save the input functional requirement prompts, and click the Execute button to execute the corresponding processing function and output the results.

[0038] In order to better monitor the status of the workflow, after clicking the production node to execute, the orange lights of the production node and the previous nodes light up. At this time, you can view the workflow execution status and resource consumption in the operation information area at the bottom of the interface, or you can exit to the home page to view the node operation status, operation records and logs, and overall dimensional monitoring. When the green light of the production node lights up, you can view the output results of the node. In one embodiment, the home page provides four pages: My Workflow, My Output Results, Workflow Templates, and Import AI Nodes. Among them, my workflow contains information such as workflow name, number of AI nodes, creation time, modification time, and operation, and the workflow execution status and history can be viewed and managed. My output results contain information such as workflow name, workflow output results, and result output time. The workflow template contains information such as saved templates, number of AI nodes, creation time, and modification time. Importing AI nodes is oriented to function providers and supports standardized and interfaced import of AI nodes.

[0039] During the execution of the production node, if you are not satisfied with the output result of a production node, you can click on the production node and edit the work form again (modify the corresponding functional requirements prompt) or replace the production node to re-create; when the production node needs to be replaced or re-edited, the output result of the previous node of the production node remains unchanged, and the subsequent node re-executes the processing process. In actual applications, if a production node in the workflow is replaced or re-edited, the indicator light of its subsequent node turns gray, indicating that it needs to be re-executed.

[0040] Step 4: After executing all nodes in the workflow, the final result is output through the output component node at the last level of the workflow.

[0041] Generally, when you need to output the results of multimedia content creation, you need to add an output component node at the end of the workflow. Through this node, you can output the final result, which can be viewed and saved in this interface or on the home page.

[0042] In this embodiment, a parallel processing solution for multiple objects / inputs is also proposed. In the process of executing all nodes of the workflow, the following processing method is adopted:

[0043] (1) If the adjacent production nodes are narrowly dependent, that is, the output quantity of the preceding production node is consistent with the input quantity of the succeeding production node, the computing tasks of the two production nodes are executed in a pipeline form on the same cloud node. When the computing task completion rate of the preceding production node reaches the set value, the image is pulled on the cloud node to build the succeeding production node. After the preceding production node is calculated, the output result is saved in the database, and the succeeding production node is notified to start the computing task. The intermediate results in the memory are shared between tasks. In one embodiment, the cloud node is implemented using a cloud server or a local server provided by a cloud vendor managed by K8S.

[0044] (2) If some production nodes are widely dependent, that is, the output quantity of the preceding production node is inconsistent with the input quantity of the succeeding production node, the preceding production node and the succeeding production node are executed on different cloud nodes. After the preceding production node completes the calculation, it saves the output result in the database and notifies the succeeding production node to read it. After reading the output files of all the preceding production nodes, the succeeding production node will execute the corresponding calculation task.

[0045] (3) If there is no dependency between the two production nodes, the computing tasks are executed in parallel on different cloud nodes.

[0046] The present invention proposes to use a workflow orchestration interface interaction to orchestrate generative artificial intelligence capabilities and professional software. Compared with the previous situation where content creators needed to switch between multiple websites, multiple systems, adapt to different interaction scenarios, and frequently import and export, it took longer time to achieve the creation of multimedia content works of normal duration and content that meets the intention. This truly realizes the flexible combination of generative artificial intelligence and professional software, and can flexibly customize the production of multimedia content.

[0047] Example 2

[0048] In order to realize various AI capabilities and flexibly define workflows, the embodiment of the present invention proposes a multimedia content creation system based on generative artificial intelligence. Figure 2, mainly includes node units and workflow orchestration units, where the node unit is used to realize the functions required for multimedia content creation; the workflow orchestration unit is mainly used to arrange nodes to form workflows.

[0049] Specifically, the node unit mainly provides input and output nodes and production nodes for selection by the workflow orchestration unit. Among them, the input and output nodes include input nodes and output component nodes. In this embodiment, the input node provides video input nodes, image input nodes, audio input nodes, text input nodes, content library input nodes, etc., which respectively realize the import of video, image, audio, text, and content library. The output component node is mainly used to output and save the output results of the previous node. The production nodes mainly include GENAI nodes and audio and video processing nodes. In this embodiment, the GENAI node mainly provides standardized management of various types of generative AI capabilities in the field of multimedia creation; audio and video processing nodes can be implemented through professional media editing software.

[0050] The workflow arrangement unit is used to select input and output nodes and production nodes in the node unit, arrange the selected nodes according to the creation requirements, establish a workflow, and complete the multimedia content creation by executing the tasks of each node in the workflow.

[0051] In this embodiment, when establishing a workflow, if external materials need to be imported, such as video, image, audio, text, and content library, corresponding input nodes can be added. If external import is not required, materials can be directly generated through production nodes. When the workflow execution results need to be output, output component nodes can be added at the end of the workflow. If output is not required, the output results of the node can be directly viewed after the corresponding node executes the task.

[0052] In one embodiment, the workflow arrangement unit provides a visual interface to complete the creation of workflows in a graphical interactive manner. In the workflow arrangement unit, different functional nodes can be added by clicking or dragging production nodes to the workflow canvas, and connection and arrangement operations can be performed according to creation requirements to create a workflow for multimedia content creation.

[0053] In the established workflow, each production node includes a work form operation interface for inputting functional requirement prompts, assisting the production node to perform corresponding processing work to generate the required multimedia content, and after each production node work is completed, the node output result can be displayed by clicking the production node. Specifically, by clicking the production node, the user can add the corresponding functional requirement prompts in the work form of the production node, which can further control the quality of multimedia content creation.

[0054] If the output result of a certain production node is not satisfied, the processing of the production node can be re-executed by re-editing the work form, or replaced with other production nodes to perform the corresponding task. In this embodiment, when a production node in the workflow needs to be replaced or re-edited, the output result of the previous node of the production node remains unchanged, and the subsequent production node needs to re-execute the processing.

[0055] It should be noted that, among the adjacent nodes of the workflow, the output of the previous node serves as the input of the next node; in this embodiment, the production nodes are divided into single-input single-output nodes, single-input multiple-output nodes, multiple-input single-output nodes, and multiple-input multiple-output nodes according to functional requirements, that is, the input of a production node can be the output of multiple production nodes, and the output of a production node can also be the input of multiple nodes.

[0056] Furthermore, in this embodiment, the multimedia content creation system also includes a display unit for displaying the output results of each production node or output component node.

[0057] In order to realize the unified call management of nodes in the node unit, in this embodiment, a unified management standard is designed for the node unit, which is oriented to the AI ​​function provider, supports the standardized and interfaced import of the required production nodes, and standardizes the management of various types of generative AI capabilities, professional media editing software and other applications in the field of multimedia creation. Specifically, in this embodiment, the node unit has interfaces such as data input, data output, data reading, message notification, and parameter modification. Among them, the data input interface includes the data source input address, data input type, and data source input quantity. The data input interface can also read data from the database according to the data source input address. The data output interface includes the data source output address and the data output type. The message notification interface includes two types: receiving messages and sending messages. The parameter modification interface supports a variety of input parameters, such as content theme, style, prompt word, output resolution, etc. Furthermore, the node unit also has global variable fields such as application name, globally unique identifier, data source input quantity, data input type, and data output type to realize node management.

[0058] Furthermore, the node unit provides node registration capabilities. Algorithms that conform to standard interfaces can be registered in the system as production nodes for calling. It provides basic field controls such as input modules, output modules, drop-down components, and text components. Work forms can be formed by graphical drag and drop. It provides resource setting capabilities and can configure the node's CPU, memory, GPU, video memory and other computing resources. After the settings are completed, the system will generate a configuration file, import it into the configuration center, and the node unit will add the node.

[0059] Since the production node supports multiple-input and multiple-output, in actual applications, the workflow may include parallel processing processes. For example, the three preceding routes perform different processing tasks and then converge to the same production node for subsequent processing. At this time, the three preceding routes can execute computing tasks in parallel.

[0060] Specifically, when the workflow executes a computing task, if the adjacent production nodes are narrowly dependent, that is, the output quantity of the preceding production node is consistent with the input quantity of the succeeding production node, the computing task is executed in a pipeline form on the same cloud node. When the completion rate of the computing task of the preceding production node reaches the set value, the image is pulled on the cloud node to build the succeeding production node. After the computing of the preceding production node is completed, the output result is saved in the database to facilitate the user to view the execution process; and the succeeding production node is notified to start the computing task, and the intermediate results in the memory are shared between tasks.

[0061] When the workflow executes computing tasks, if the adjacent production nodes are widely dependent, that is, the output quantity of the preceding production node is inconsistent with the input quantity of the succeeding production node (for example: the upstream is a single-input single-output node, and the downstream is a multi-input single-output or multi-input multi-output node; the upstream is a single-input multi-output node, and the downstream is a single-input single-output or single-input multi-output node), then the preceding production node and the succeeding production node are executed on different cloud nodes. After the preceding production node completes the calculation, the output result is saved in the database, and the succeeding production node is notified to read it. When all the output files of the preceding production nodes are read, the succeeding production node executes the corresponding computing task.

[0062] In order to improve computing efficiency, production nodes that do not have dependencies can be computed in parallel on different cloud nodes at the same time. In one embodiment, the cloud node can be implemented using a cloud server or a local server provided by a cloud vendor managed by K8S.

[0063] In one embodiment, the multimedia content creation system also includes a status monitoring unit for displaying the real-time execution status and resource consumption of the workflow; as well as viewing the node operation status, operation records and logs, and overall dimensional monitoring.

[0064] In a preferred embodiment, the multimedia content creation system also includes a work template unit, a user workflow management unit, and an output result management unit. The workflow template is used to provide a saved workflow template, including input and output nodes, production nodes, scheduling relationships, creation time, modification time and other information. The user workflow management unit contains information such as workflow name, number of AI nodes, creation time, modification time, operation, etc., and can view and manage the workflow execution status and history. The output result management unit contains information such as workflow name, workflow output result, result output time, etc.

[0065] The multimedia content creation system of generative artificial intelligence proposed in this embodiment uses a graphical interactive method to build a workflow based on multiple types of AI nodes, supports the simultaneous processing of multiple objects such as input videos, images, texts, etc., flexibly configures the required functional nodes, supports single-input single-output nodes, single-input multiple-output nodes, multiple-input single-output nodes, and multiple-input multiple-output nodes; intuitively view the multimedia content creation effect and final result of each node, and ensure the controllable generation of generative artificial intelligence multimedia content. In addition, it supports the import of AI capability nodes provided by different developers through standardized interfaces to achieve low-threshold and low-cost functional expansion.

[0066] In order to more clearly explain the generative artificial intelligence multimedia content creation system proposed in the present invention, the system is further described by the creation and working process of a workflow. Figure 3 In this example, the production nodes include but are not limited to AI function nodes such as text-to-image, text-to-video, intelligent cartoonization, intelligent replacement, image-to-video, transition video generation, video interpolation, object motion generation, photo digital human driving, video airspace expansion, background replacement, style transfer, video color grading, and green screen keying.

[0067] 1. Add a video input node and list it as source 1. At the same time, add two production nodes: text to video and text to image.

[0068] 2. Click the text to generate a video node, list it as source 2, edit the positive prompt words and negative prompt words in the work form operation interface, set the width, height settings, and video length settings, and click Save or execute the node calculation process.

[0069] 3. Click the text to generate a picture node, list it as source 3, edit the positive and negative text prompts on the right side of the work form operation interface, set the width and height settings, and click Save or execute the node calculation process.

[0070] 4. Create two new smart cartoonization nodes, link the input video and text generation video nodes respectively, edit the two smart cartoonization nodes through the work form operation interface, including cartoonization style selection, etc. Click to save or execute the node calculation process.

[0071] 5. If you are not satisfied with a certain segment of the video after the intelligent cartoonization processing of source 2, you can link or add an intelligent replacement node, add the replacement content in the work form operation interface of the node to replace it, and click Save or execute the calculation process of the node.

[0072] 6. If there are other requirements for the style of source 3, create a new style transfer node, link the text to image node, enter the ideal style image in the work form operation interface of the node, edit the positive and negative prompt words in the text, set the width and height settings, and click Save or execute the calculation process of the node.

[0073] 7. After processing source 1 and source 2 separately, generate a video transition with satisfactory results. Create a new transition video generation node, link the previous processing results, and click Save or execute the node calculation process.

[0074] 8. After establishing the style of source 3, you need to generate a video from the image. You can add an image to video node, link the previous result of source 3, and click Save or execute the calculation process of the node.

[0075] 9. Perform video color grading on the composite result of source 1 and source 2 to unify the style with source 3. Then, add a video color grading node and link the transition video production node. Click Save or execute the node calculation process.

[0076] 10. Finally, add a multi-video object synthesis node, link the video color grading and image generation video nodes at the same time, synthesize the above results into multiple video objects, edit the order and requirements in the work form operation interface of this node, and then link the output component node. Click Save or execute the calculation process of this node. After execution, you can view the full workflow effect and save the file.

[0077] For those skilled in the art, the specific meanings of the above terms in the present invention can be understood in specific situations; the drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0078] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for creating multimedia content using generative artificial intelligence, characterized in that: include: In the workflow arrangement interface, click or drag to add the required production nodes and input and output nodes to the workflow canvas; wherein, the workflow arrangement interface provides a workflow canvas and the input and output nodes and production nodes required for creation. The production nodes are oriented to AI function providers and are imported by standardized interfaces to provide various types of multimedia content creation functions based on generative artificial intelligence and professional media editing software; Arrange connections between nodes according to the required multimedia content creation process to create a multimedia content creation workflow; After clicking a production node, the work form operation interface of the production node pops up, and the corresponding functional requirement prompts are added in the work form according to the functions of the production node, and the corresponding processing functions are executed and the results are output; After executing all nodes in the workflow, the final result is output through the output component node at the last level of the workflow; During the execution of all nodes of the workflow, if the output quantity of the preceding production node is consistent with the input quantity of the succeeding production node, the computing tasks of the two production nodes are executed in a pipeline form on the same cloud node. When the computing task completion rate of the preceding production node reaches the set value, the image is pulled on the cloud node to build the succeeding production node. After the preceding production node is calculated, the output result is saved in the database, and the succeeding production node is notified to start the computing task. The intermediate results in the memory are shared between tasks. If the output quantity of the preceding production node is inconsistent with the input quantity of the succeeding production node, the preceding production node and the succeeding production node are executed on different cloud nodes. After the preceding production node is calculated, the output result is saved in the database, and the succeeding production node is notified to read it. When all the output files of the preceding production nodes are read, the succeeding production node executes the corresponding computing task. If there is no dependency between the two production nodes, the computing tasks are executed in parallel on different cloud nodes.

2. The method for creating multimedia content using generative artificial intelligence according to claim 1, characterized in that: When you are not satisfied with the output result of a production node, you can click on the production node and re-edit the work form or replace the production node to re-create it; when a production node needs to be replaced or re-edited, the output result of the preceding node of the production node remains unchanged, and the subsequent node re-executes the processing process.

3. A multimedia content creation system based on generative artificial intelligence, characterized in that: include: A node unit, including input and output nodes and production nodes, for selection by a workflow arrangement unit; wherein the input and output nodes include video input nodes, image input nodes, audio input nodes, text input nodes, content library input nodes, and output component nodes; the production node is used to provide various types of multimedia content creation functions based on generative artificial intelligence and professional media editing software; the node unit is oriented to AI function providers and supports standardized and interfaced import of production nodes; the node unit specifically includes a data input interface, a data output interface, a message notification interface, and a parameter modification interface; wherein the data input interface includes a data source input address, a data input type, and a data source input quantity, and the data input interface simultaneously reads data from a database according to the data source input address; the data output address includes a data source output address and a data output type; the message notification interface includes receiving messages and sending messages; the parameter modification interface supports a variety of input parameters, including content themes, styles, prompt words, and output resolution; The workflow arrangement unit is used to select the input and output nodes and production nodes in the node unit, and arrange the selected nodes in the workflow canvas according to the creation needs to establish a workflow. By executing the tasks of each node in the workflow, multimedia content creation can be completed.

4. The multimedia content creation system of generative artificial intelligence according to claim 3, characterized in that: Each production node includes a work form operation interface for inputting functional requirement prompts to assist the production node in performing corresponding processing work to generate the required multimedia content. After the work of each production node is completed, the node output result can be displayed by clicking the production node.

5. The multimedia content creation system of generative artificial intelligence according to claim 3 or 4, characterized in that: When a production node in the workflow needs to be replaced or re-edited, the output result of the predecessor node of the production node remains unchanged, and the subsequent node re-executes the processing process; among them, the secondary editing means that the output result of the production node does not meet the requirements, and the work form needs to be re-edited to re-execute the production node processing process.

6. The multimedia content creation system of generative artificial intelligence according to claim 3, characterized in that: In the workflow orchestration unit, if the output quantity of the preceding production node is consistent with the input quantity of the succeeding production node, the computing tasks of the two production nodes are executed in a pipeline form on the same cloud node. When the completion rate of the computing task of the preceding production node reaches a set value, the image is pulled on the cloud node to build the succeeding production node. After the computing of the preceding production node is completed, the output result is saved in the database, and the succeeding production node is notified to start the computing task. The intermediate results in the memory are shared between tasks.

7. The multimedia content creation system of generative artificial intelligence according to claim 3, characterized in that: In the workflow orchestration unit, if the output quantity of the preceding production node is inconsistent with the input quantity of the succeeding production node, the preceding production node and the succeeding production node are executed on different cloud nodes. After the preceding production node completes the calculation, the output result is saved in the database and the succeeding production node is notified to read it. Only after all the output files of the preceding production nodes are read, the succeeding production node executes the corresponding calculation task.

8. The multimedia content creation system of generative artificial intelligence according to claim 3, characterized in that: In the workflow orchestration unit, if there is no dependency between two production nodes, computing tasks can be executed in parallel on different cloud nodes.