Skill module generation method for video generation and corresponding apparatus

Abstract

Embodiments of the present application disclose a skill module generation method and device for video generation, the method comprising: displaying a user interface, the user interface comprising a list area and a canvas area, the list area displaying a plurality of atomic capability units, each atomic capability unit corresponding to an agent node having a predefined input / output data structure, and being used to execute a video generation function; in response to a visual arrangement operation on a target atomic capability unit, displaying an arrangement result of combining the target atomic capability unit into an ordered workflow in the canvas area; and in response to a skill generation operation, generating a skill module based on the ordered workflow, the skill module being an executable software component encapsulating a topological data structure of the ordered workflow, the topological data structure comprising data dependency relationships and execution sequences, and being used to call corresponding agent nodes to generate a video. Through visual arrangement and automatic generation of the skill module, the dependence on technical knowledge of the user is reduced, and the efficiency of generating a video creative is improved.

Description

Technical Field

[0001] This application relates to the fields of video processing and artificial intelligence in the field of next-generation information technology, and in particular to a method and corresponding apparatus for generating skill modules for video generation. Background Technology

[0002] In the field of AIGC (Artificial Intelligence Generated Content), with the rapid evolution of large language models and multimodal generation capabilities, developers are quickly realizing video ideas by building workflows for video generation. However, traditional implementation methods require a high level of knowledge from the workflow builders. Typically, developers with specialized expertise write code to connect different intelligent agent nodes for specific creative needs, defining parameter passing rules and execution order for each intelligent agent node, ultimately forming a fixed, executable workflow configuration. This workflow configuration method, which relies on manual development, requires developers to have a deep understanding of the details of each intelligent agent node. Adjustments to the process and the addition of new nodes all require reinvestment of development resources, resulting in low efficiency in realizing creative solutions. Summary of the Invention

[0003] In view of this, this application provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating skill modules for video generation, in order to improve the efficiency of generating video ideas.

[0004] This application provides the following solution: According to a first aspect, a method for generating skill modules for video generation is provided, the method comprising: The user interface is displayed, which includes a list area and a canvas area. The list area displays multiple atomic capability units, each of which corresponds to an agent node and has a predefined input data structure definition and output data structure definition, used to perform video generation-related functions. In response to the visualization and arrangement operation of the target atomic capability unit among the plurality of atomic capability units, the arrangement result of the target atomic capability unit is displayed in the canvas area, and the arrangement result is used to combine the target atomic capability units into an ordered workflow; In response to the operation that triggers skill generation, a skill generation request is sent to request the generation of a skill module based on the ordered workflow. The skill module is an executable software component that encapsulates the topology data structure of the ordered workflow. The topology data structure includes the data dependencies and execution order between the target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

[0005] According to a second aspect, a method for generating skill modules for video generation is provided, the method comprising: Multiple atomic capability units are provided, each of which corresponds to an agent node and has a predefined input data structure definition and output data structure definition, for performing video generation-related functions; In response to a visual orchestration operation on a target atomic capability unit among the plurality of atomic capability units, the target atomic capability units are combined into an ordered workflow based on the visual orchestration operation; In response to a skill generation request, a skill module is generated based on the ordered workflow. The skill module is an executable software component that encapsulates the topology data structure of the ordered workflow. The topology data structure includes the data dependencies and execution order between the target atomic capability units. The skill module is used to call the corresponding agent node to generate video according to the data dependencies and execution order.

[0006] According to a third aspect, a skill module generation apparatus for video generation is provided, the apparatus comprising: The interface display unit is configured to display a user interface, which includes a list area and a canvas area. The list area displays multiple atomic capability units, each of which corresponds to an agent node and has a predefined input data structure definition and an output data structure definition, used to perform video generation-related functions. An operation monitoring unit is configured to monitor operations on the user interface. The interface display unit is further configured to, in response to the operation monitoring unit detecting a visual arrangement operation on a target atomic capability unit among the plurality of atomic capability units, display the arrangement result of the target atomic capability unit in the canvas area, wherein the arrangement result is used to combine the target atomic capability units into an ordered workflow; The request sending unit is configured to send a skill generation request in response to the operation listening unit detecting an operation that triggers skill generation. The skill module is an executable software component that encapsulates the topology data structure of the ordered workflow. The topology data structure includes the data dependencies and execution order between the target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

[0007] According to a fourth aspect, a skill module generation apparatus for video generation is provided, the apparatus comprising: An atom providing unit is configured to provide a plurality of atomic capability units, each of which corresponds to an agent node and has a predefined input data structure definition and an output data structure definition, for performing video generation-related functions; A workflow composition unit is configured to respond to a visual orchestration operation on a target atomic capability unit among the plurality of atomic capability units, and to combine the target atomic capability units into an ordered workflow based on the visual orchestration operation, wherein the ordered workflow defines the data dependencies and execution order between the target atomic capability units; The skill generation unit is configured to generate a skill module based on the ordered workflow in response to an operation that triggers the generation of a skill. The skill module is an executable software component that encapsulates the topological data structure of the ordered workflow and is used to call the corresponding agent node to generate video according to the data dependencies and execution order.

[0008] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first and second aspects above.

[0009] According to a sixth aspect, an electronic device is provided, comprising: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first and second aspects above.

[0010] According to a seventh aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described in any one of the first and second aspects above.

[0011] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application transforms manually developed, hard-coded complex logic into a structured, orchestratable workflow that can be automatically generated and executed by the system. This is achieved by predefining each corresponding agent node as an atomic capability unit with a standard input / output data structure and providing a visual interface to respond to visual orchestration operations targeting these atomic capability units. This workflow is then encapsulated as an executable software component, forming reusable skill modules. This visual orchestration interface and automated skill module generation mechanism allow users to construct video generation workflows and encapsulate them as standardized executable software components through simple visual orchestration without writing code. This reduces reliance on user technical knowledge and improves the efficiency of generating video ideas.

[0012] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a system architecture diagram applicable to the embodiments of this application.

[0015] Figure 2 A flowchart of a skill module generation method provided in an embodiment of this application.

[0016] Figure 3 This is a schematic diagram of a user interface provided for an embodiment of this application.

[0017] Figure 4 This is a schematic diagram of a skills list page provided in an embodiment of this application.

[0018] Figure 5 This is a schematic diagram of a conversation page with a video production AI assistant provided in an embodiment of this application.

[0019] Figure 6 This is a schematic diagram showing the execution phase and execution status on a session interface, provided for embodiments of this application.

[0020] Figure 7 A flowchart of another skill module generation method provided in an embodiment of this application.

[0021] Figure 8 This is a schematic block diagram of a skill module generation device provided in an embodiment of this application.

[0022] Figure 9 This is a schematic block diagram of another skill module generation device provided in an embodiment of this application.

[0023] Figure 10 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0027] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0028] In the field of AIGC (AI Generative Content Generation) technology, the common approach to rapidly generate video creative ideas is as follows: Developers with specialized knowledge write code to connect different intelligent agent nodes for specific creative needs, defining parameter passing rules and execution order for each node, ultimately forming a fixed, executable workflow configuration. This workflow configuration method, relying on manual development, requires developers to have a deep understanding of the details of each intelligent agent node. Adjustments to the process and the addition of new nodes all require reinvestment of development resources, resulting in low efficiency in realizing creative solutions.

[0029] When this solution is applied to scenarios requiring rapid response to changing market demands and where creative arrangement needs to be led by non-R&D roles (such as product managers or operations personnel), its performance is less than ideal. Through in-depth analysis, the inventors discovered that the root cause of these shortcomings lies in the fact that each new video creative requirement necessitates the construction of a completely new, hard-coded execution chain, lacking a standardized middleware layer to decouple capability definition from capability arrangement. Furthermore, the invocation and combination of intelligent agent nodes, as well as the adjustment of parameters in the process definition, all require writing program code, which sets an extremely high barrier to entry for business personnel without programming skills.

[0030] In view of this, this application provides a new approach. To facilitate understanding of this application, the system architecture on which this application is based will first be described. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture may include: a client and a server.

[0031] The server side and the client side are the two main parts of an application service. The server side uses a server as the main hardware infrastructure and may include one or more software service modules, forming a front-end and back-end that work together with the client side.

[0032] The user terminal can include, but is not limited to, smart mobile terminals, wearable devices, and PCs (Personal Computers). Smart mobile devices can include mobile phones, tablets, PDAs (Personal Digital Assistants), and connected car terminals. Wearable devices can include smartwatches, smart glasses, smart bracelets, VR (Virtual Reality) devices, AR (Augmented Reality) devices, and mixed reality devices (devices that support both virtual and augmented reality), etc.

[0033] The user end can be an application located on the user terminal, or it can be a plugin or software development kit (SDK) or other functional unit set in the application.

[0034] A server can be a standalone server, a server cluster, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a hosting product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0035] It should be understood that Figure 1 The server and client shown are merely illustrative. Depending on implementation needs, there can be any number of server and client components.

[0036] Figure 2 This is a flowchart of a skill module generation method for video generation provided in an embodiment of this application. This method can be... Figure 1 The user-side execution in the system shown is as follows. Figure 2 As shown, the method may include the following steps: Step 201: Display the user interface, which includes a list area and a canvas area. The list area displays multiple atomic capability units. Each atomic capability unit corresponds to an agent node and has predefined input data structure definitions and output data structure definitions, which are used to perform video generation-related functions.

[0037] Step 203: In response to the visual orchestration operation of the target atomic capability unit among multiple atomic capability units, display the orchestration result of the target atomic capability unit in the canvas area. The orchestration result is used to combine the target atomic capability units into an ordered workflow.

[0038] Step 205: In response to the operation that triggers skill generation, a skill generation request is sent to request the generation of a skill module based on an ordered workflow. The skill module is an executable software component that encapsulates a topological data structure of an ordered workflow. The topological data structure includes the data dependencies and execution order between target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

[0039] As can be seen from the above process, this application predefines each corresponding agent node as an atomic capability unit with a standard input / output schema (data structure definition) and provides a visual interface to respond to visual orchestration operations for these atomic capability units. This transforms manually developed, hard-coded complex logic into a structured, orchestratable workflow that can be automatically generated and executed by the system. This workflow is then encapsulated into executable software components, forming reusable skills. This visual orchestration interface and automated skill module generation mechanism allow users to build video generation workflows and encapsulate them into standardized executable software components through simple visual orchestration without writing code. This reduces reliance on user technical knowledge and improves the efficiency of generating video ideas.

[0040] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments. It should be noted that the terms "first," "second," etc., used in this disclosure do not have limitations in terms of size, order, or quantity, but are only used to distinguish them by name. For example, "first component" and "second component" are used to distinguish two components by name.

[0041] First, the above step 201, namely "displaying the user interface, which includes a list area and a canvas area, and the list area displays multiple atomic capability units", will be described in detail with reference to the embodiments.

[0042] An atomic capability unit (AMU) is a visual representation of the smallest functional unit that can be independently selected and manipulated on a user interface. It can be understood as a graphical entry point for a functional node, behind which lies an agent node encapsulating a specific video generation subtask. Each AMU corresponds to an agent node and has predefined input data structures and output schemas for executing video generation-related functions.

[0043] Intelligent agent nodes can be pre-developed by researchers, with input and output schemas defined and NodeTemplateIds (node ​​identifiers) registered. Their capabilities are standardized and reusable. The pre-defined input and output schemas clearly specify the format and fields of the data the intelligent agent node needs to receive when executing a task, as well as the format and fields of the data it will output after execution. This lays the foundation for the orderly and correct transfer of data between nodes in subsequent workflows.

[0044] In this embodiment, the intelligent agent node may include, but is not limited to, input-type intelligent agents, processing-type intelligent agents, generation-type intelligent agents, understanding-type intelligent agents, and output-type intelligent agents. Specifically, processing-type intelligent agents include at least one of the following: material retrieval intelligent agent, digital human selection intelligent agent, TTS (Text To Speech) selection intelligent agent, BGM (Background Music) selection intelligent agent, and sticker selection intelligent agent. Generation-type intelligent agents may include at least one of the following: script generation intelligent agent, text-to-image intelligent agent, and text-to-video intelligent agent. Understanding-type intelligent agents include at least one of the following: image understanding intelligent agent, audio understanding intelligent agent, and text understanding intelligent agent.

[0045] like Figure 3 As shown, the user interface includes a list area and a canvas area. The list area displays entries for multiple atomic capability units, each entry including a functional description or functional label for the corresponding agent node. The functional description is parsed from the agent node's metadata, and the functional label is derived from the functional description.

[0046] For example, a node named "Material Retrieval Agent" might have a description field in its metadata that reads, "Retrieves high-quality videos from first-level and second-level industries, returning only one high-quality video for analysis and video element generation." When constructing atomic capability units, the server automatically parses this description from the metadata and presents it directly to the user as the entry's explanatory text. This allows users to quickly understand the purpose of each node simply by reading its natural language description, without needing to delve into its internal implementation.

[0047] Furthermore, the server can generate a more concise functional label based on this functional description through keyword extraction, semantic abstraction, or preset mapping rules. For example, the description could be abstracted as "Search for high-quality videos" or "Material retrieval." This label can be attached to the entry in a prominent manner (such as a marker, colored icon, or prefix) so that users can quickly discover and understand the function of the atomic capability unit. Through this display of entries with semantic information, users can quickly filter out atomic capability units that meet their current creative needs before visual arrangement, thereby greatly shortening the preparation time for workflow construction and improving the intuitiveness and efficiency of arrangement.

[0048] Of course, the above display methods can be adjusted according to the actual deployment scenario and are not the only ones. As an alternative implementation, instead of parsing the description in the metadata, the original name of the agent node can be displayed directly, with detailed description information or functional tags only popping up when the user hovers over or clicks on the item, thus saving list space. Another alternative is that the functional tags are not automatically generated by the system, but are manually specified by the R&D personnel maintaining the atomic capability units when registering the nodes, and the system directly displays them as functional tags. In addition, a tag generator based on a large language model can be introduced, which automatically reads the input-output schema and functional description when the atomic capability unit is registered, and generates a set of highly generalized, standardized functional tags. These alternative solutions all achieve the core purpose of helping users quickly locate atomic capability units and balance automation, semantic accuracy, and resource overhead at different levels.

[0049] The following describes in detail step 203, namely, "in response to the visual arrangement operation of the target atomic capability unit among multiple atomic capability units, the arrangement result of the target atomic capability unit is displayed in the canvas area, and the arrangement result is used to combine the target atomic capability units into an ordered workflow," with reference to the embodiments.

[0050] Visual orchestration refers to the interactive behavior of users building or defining workflows by directly manipulating visual elements on the user interface. This is significantly different from the textual approach of defining workflows through code. Visual orchestration can include a series of coherent, intuitive touch or mouse operations to ensure that users can efficiently deploy atomic capability units from the list area to the canvas area and establish logical relationships between them.

[0051] As a specific implementation, this visual arrangement operation can include dragging atomic capability units from the list area to the canvas area, as well as connecting and arranging elements representing different atomic capability units in the canvas area. The essence of these interactions received by the user interface is to obtain the user's intent regarding how to combine and connect the various atomic capability units.

[0052] Specifically, in response to a user's action of pressing and holding a target atomic capability unit in the list area (e.g., long-pressing an item on a touchscreen, or left-clicking and holding the button), the system will immediately display a visual element corresponding to the target atomic capability unit near the pressed location. This visual element can be a semi-transparent card that matches the appearance of the list item, or a simplified node icon. Its purpose is to provide the user with immediate feedback, clearly informing them of the object currently being manipulated and indicating the graphical component to be placed. Through this initial visual feedback, the user can quickly confirm whether their selected target is correct, thereby avoiding accidental operations.

[0053] Following this, in response to the user continuing the press-and-hold operation and performing a drag-and-drop operation from the list area to the canvas area (e.g., moving the cursor or touch point from the list area to any position within the canvas area without releasing the finger or mouse), the system dynamically renders the visual element in real time, following the user's movement trajectory. When the user drags the visual element to the canvas area and releases it, the system, in response to the end of the drag operation, fixes the visual element at the end position of the drag operation on the canvas area. At this point, a node instance of an atomic capability unit has been successfully placed on the canvas. The user can continue to repeat this operation to deploy multiple atomic capability units sequentially onto the canvas.

[0054] Finally, within the canvas area, the user needs to establish a data flow relationship between the two placed visualization elements. In response to the user's action of connecting two visualization elements in the canvas area (e.g., dragging a line from one node's output port to another node's input port and releasing it, or clicking the starting node once and then the ending node again), the user sets a directed edge between the two visualization elements in the canvas area. This directed edge is visually represented as a line with an arrow, and it is recorded as a data structure containing source node identifiers and target node identifiers. This directed edge is a graphical representation of the data dependencies and execution order between the target atomic capability units: the arrow direction indicates the direction of the data flow, and the system defaults to the source node's execution order taking precedence over the target node.

[0055] The presented visual arrangement results can be as follows Figure 3 As shown, through the three-step interaction process of "press-drag-connect," users can intuitively complete the workflow construction without any complex menu operations or parameter input. This interaction method greatly reduces the usage threshold for non-technical users, enabling business personnel to quickly assemble complex video production pipelines like building blocks.

[0056] Of course, the above-described interactive gestures are merely a preferred embodiment and do not constitute a limitation of the present invention. As an alternative overall solution, the system can adopt a "click to select - then click to place" mode: the user clicks to select an atomic capability unit in the list area, and then clicks at a specific location in the canvas area; the system then places the unit's visual element at that location. This approach avoids long-press and drag operations, has lower precision requirements on touchscreens, and is suitable for mobile scenarios. For connections between nodes, a method of "first selecting the source node, then popping up a list of all connectable target nodes, and the user clicking to select" can be used instead of directly dragging and dropping connections, thereby reducing the requirements for precise operation.

[0057] Some technical features can also be replaced. For example, displaying visual elements near the clicked location can be replaced by simply highlighting items in the list without generating independent visual elements until the user drags and drops them onto the canvas area. Connecting two visual elements can be done by selecting both nodes simultaneously and clicking the "Connect" button in the toolbar, in addition to dragging and dropping. These alternatives achieve the core purpose of creating directed edges and can be flexibly adapted to the operating habits and device types of different user groups to ensure the universality and ease of use of the orchestration scheme.

[0058] To ensure the logical correctness and executability of workflows built through visual orchestration in the data flow direction, and to avoid subsequent execution failures due to invalid connections, this application introduces a dependency constraint verification mechanism when setting directed edges between two visual elements. Specifically, when a user performs an operation to connect two visual elements in the canvas area, the system does not establish a connection unconditionally, but first determines whether the atomic capability units represented by the two elements satisfy predefined dependency constraints. The core of this dependency constraint is that the output data structure definition of the atomic capability unit as the source node must be able to match the input data structure definition of the atomic capability unit as the target node. For example, if the output of one atomic capability unit is an "image file address" (e.g., the output schema is defined as type `string` and the format is `imageUrl`), while the input requirement of another atomic capability unit is an "audio file address" (e.g., the input schema is defined as type `string` and the format is `audioUrl`), the system will determine that the two are incompatible and the types do not match. In addition, dependency constraints may include checking whether there is a circular dependency (i.e., whether there is a path that starts from the target node, goes through a series of edges, and returns to the source node) to ensure that the entire workflow forms a directed acyclic graph (DAG).

[0059] If the verification results show that all dependency constraints are satisfied between the two visual elements, the system sets the directed edge normally in the canvas area and updates the internal workflow topology data structure. If the verification results show that dependency constraints are not satisfied (e.g., output and input are incompatible, or a circular dependency may be formed), the system does not establish the connection on the canvas and displays a prompt message on the user interface. This prompt message indicates at least one of the following: the directed edge setting failed and the reason for the failure. For example, "Type mismatch: the source node output is an image type, the target node input requires a text type" or "This connection will cause a circular dependency, please adjust the connection order." Through this immediate and local feedback, users can quickly understand the root cause of the ineffective operation and adjust the orchestration strategy accordingly, thereby avoiding the exposure of errors only during trial operation or formal execution, significantly improving the success rate of workflow orchestration and the user experience.

[0060] It should be noted that the specific content of the above dependency constraints can be adjusted according to the system implementation. For example, the constraints can only check type compatibility, or they can further check format, enumeration value range, or business semantics. Furthermore, there are several alternatives for the timing and method of verification. Regarding overall alternatives, the system can also forgo real-time verification and instead perform batch verification after the user completes all connection operations, highlighting all invalid edges. Some feature alternatives include: setting a failure message for non-compliance constraints, instead of allowing users to establish connections at all times, but displaying edges that do not meet the constraints with dashed lines or special colors, and issuing a centralized warning before workflow execution; or, the system can provide an intelligent connection recommendation function, where when a user attempts to connect two nodes, the system automatically pops up a candidate list, displaying only available ports that meet the constraints, allowing the user to choose from them, thus fundamentally avoiding invalid connections. These alternatives all achieve the core objective of ensuring the correctness of workflow logic, but each has its own trade-offs regarding the immediacy of interaction and user freedom.

[0061] To further enrich users' interactive control capabilities during the visual orchestration process, such as Figure 3 As shown in the illustration, this embodiment introduces a separate control area in the user interface. This control area is used to display and configure detailed information about the currently selected visual element.

[0062] In a preferred embodiment, in response to a user's selection of a target visual element in the canvas area (e.g., by clicking the visual element or tapping it on a touchscreen), the client requests all configurable parameters of the atomic capability unit corresponding to the target visual element from the server. These configurable parameters are derived from non-required fields in the input schema defined during registration of the atomic capability unit, as well as some additional runtime configuration items. Subsequently, the system displays the obtained parameters and their current default values ​​or previously set values ​​in the control area.

[0063] This control area can be located on the right side, bottom, or as a floating panel of the user interface, and its size and position can be adjusted according to user preferences. For example, when a user selects a visual element corresponding to a TTS agent node on the canvas, the control area will dynamically load and display the node's parameters, such as "voice," "speed," and "volume." Each parameter is equipped with a corresponding input box, drop-down menu, or slider, making it easy for the user to modify directly.

[0064] Furthermore, in response to a user's modification of any configurable parameter in the control area (e.g., switching the "Timbre" dropdown menu from "Default" to "Sweet Female Voice," or adjusting the "Volume" slider from 80% to 100%), an update request is immediately generated and sent to the server. Upon receiving this update request, the server synchronously updates the values ​​of the configurable parameters of the agent node corresponding to the target visual element. This update is reflected in the node's attributes on the canvas in real time, ensuring that the workflow definition includes the precise parameters of all nodes after customized adjustments when the user completes the orchestration. In this way, users can not only build the workflow framework macroscopically but also fine-tune the specific behavior of each functional node microscopically, effectively combining standardized components with personalized needs, thereby significantly improving the accuracy and flexibility of the orchestration results.

[0065] There are several alternative implementations for the aforementioned control area. As a holistic alternative, the user interface can forgo a fixed control area and instead, in response to user selection, pop up a floating parameter editing window near the currently selected visual element. This window is similar to a tooltip but offers more comprehensive functionality; after the user modifies the parameters, closing the window saves the changes. This approach has the advantage of not occupying additional interface space, making it suitable for devices with limited screen size. Another holistic alternative is to integrate the parameter configuration functionality into the "Property Inspector" after workflow generation. Users can then centrally view and edit the parameters of all nodes through the sidebar or bottom panel.

[0066] For certain technical features, alternatives can be made. For example, the timing of requesting and displaying configurable parameters in the control area in response to a selection operation can be delayed, only displaying them when the user explicitly triggers the parameter configuration command through double-clicking or right-clicking, rather than displaying them immediately upon clicking, to avoid interfering with the user's continuous drag-and-drop operations. Regarding sending update requests to update the values ​​of configurable parameters, a local-first optimization strategy can be adopted: the user first temporarily stores the modified parameters in a local cache, and only sends the modified results to the server in batches when the user clicks "save" or submits the workflow, thereby reducing the frequency of network requests. In addition, for parameter display, it is not limited to reading the definition of atomic capability units; default values ​​can also be loaded from the historical running configuration of the node or user personalized settings to provide a more intelligent default configuration. All of the above alternatives serve the same core purpose: to allow users to customize the behavior of each atomic capability unit in the workflow in a low-cost interactive manner.

[0067] Additionally, other parameters can be configured in the control area, such as the workflow name. Figure 3As shown, the workflow name can be configured as "Video Workflow A" in the control area.

[0068] The following describes step 205, namely, "in response to the operation of triggering skill generation, sending a skill generation request to request the skill generation module based on an ordered workflow," in detail with reference to the embodiments.

[0069] A skill module is an executable software component that encapsulates an ordered workflow topology data structure. The topology data structure includes the data dependencies and execution order between target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

[0070] After a user constructs an ordered workflow that defines the data dependencies and execution order between atomic capability units through visual orchestration operations, a specific control can be set on the user interface, such as a button labeled "Generate Skill". In response to the user's action of triggering skill generation on this control (e.g., clicking the button), the client sends a skill generation request to the server. The core payload of this request is a complete description of the ordered workflow that has been created on the current canvas.

[0071] Upon receiving the request, the server executes a packaging and encapsulation process. It first serializes and structures the received ordered workflow definition, extracting its core component: the topology data structure. This topology data structure is not simply a list of nodes or edges, but contains all the information needed by the execution engine: a unique identifier for each node, dependencies between nodes, parameters required for node execution and their values, and the global initial context definition of the workflow. This structured, machine-readable topology data structure is then embedded into a pre-defined software component framework, generating a skill module that can be independently deployed and invoked.

[0072] The skill module is essentially an executable software component. It no longer relies on raw, graphical canvas information, but instead solidifies the workflow logic programmatically. When this video generation task needs to be executed, simply calling the skill module's standardized interface (e.g., passing in a JSON object containing user scripts, images, etc.) allows the skill module to automatically parse the execution order based on its internally encapsulated topological data structure and call the corresponding underlying intelligent agent nodes accordingly, ultimately generating a complete video. For example, a workflow previously built by dragging and dropping "Script Generation," "Text-to-Image Generation," "Image-to-Video Generation," and "Video Rendering" nodes will have its internal DAG JSON parsed by the execution engine after the skill module is generated. The system will first call the script generation node, passing its output to the text-to-image generation node, then the image-to-video generation node, and finally the rendering node. All steps are executed automatically according to the definition, requiring no manual intervention. Through this "orchestrated once, reused continuously" mechanism, this application transforms users' video ideas into standardized software assets that can be directly called by machines, greatly accelerating the efficiency of video idea generation.

[0073] To further ensure the correctness and predictability of user-manufactured workflows at the execution level, this application adds an interactive component for triggering the trial run function to the control area of ​​the user interface. This control area can be the aforementioned sidebar for displaying parameters, or a dedicated function toolbar. Specifically, this control area displays the first component for triggering the trial run workflow, such as... Figure 3 As shown, for example, there is a button labeled "Run the entire workflow". The function of this first component is to allow users to conduct actual testing of the execution logic of the entire orchestration scheme in a low-risk and low-cost manner before formally submitting the workflow or generating skill modules.

[0074] In response to the user's action that triggers the first component (e.g., clicking on such component) Figure 3 As shown by the "Run Entire Workflow" button, the system immediately sends a trial run request for the ordered workflow being orchestrated on the current canvas area. This request contains the complete topology definition of the workflow and all parameters set by the user. Upon receiving this request, the server provides it to the execution engine as a high-priority instantaneous task for execution. Unlike formal execution, the trial run task typically does not persist the results to the production database, nor does it trigger subsequent release or distribution processes. Its purpose is merely to simulate a real execution environment to verify whether the workflow path is correct and whether data transfer between nodes is successful.

[0075] After the execution engine completes the scheduling and execution of all nodes in the workflow, it returns a detailed trial run result to the user terminal, which displays the result on the user interface. This trial run result is not merely a final generated video file, but a structured data set containing a snapshot of the entire workflow execution. Specifically, the result includes the execution status of each atomic capability unit in the ordered workflow (e.g., "success" marked in green, "failure" marked in red, and "executing" marked in yellow) and the output data generated by each node (e.g., the image address generated by the Wensheng Graph Agent node, the audio file address generated by the TTS Selection Agent node, and the text content output by the script generation node). The user terminal can visualize these results on a canvas area: a status icon is overlaid on the visualization element corresponding to each node; clicking the status icon allows viewing its detailed output data. This approach allows users to troubleshoot logical errors or unreasonable parameter configurations in the workflow step by step, much like debugging a program, thereby quickly locating problems, adjusting solutions, and re-verifying.

[0076] By introducing this trial operation mechanism, this invention effectively solves the uncertainty caused by the lack of real-time feedback during the orchestration process. Users can verify the feasibility and advantages / disadvantages of multiple orchestration schemes at low cost during the creative stage without having to go through the entire lengthy "orchestration-generation-approval-release" process, greatly improving iteration efficiency. In addition, since the trial operation results provide intermediate data for each node, users can clearly understand the data transformation process between each stage.

[0077] To improve the complete lifecycle management of skill modules from generation to deployment, this application provides a convenient publishing and access mechanism in the user interface. Specifically, after a user generates a skill module through the aforementioned steps, the user interface also displays a second component that triggers skill publishing, such as... Figure 3 The "Release and Go" button is shown in the image. The function of this second component is to allow users to formally submit skill modules that are still in draft or private status to the production environment, making them standardized capabilities that can be used by other users.

[0078] In response to a user's action triggering this second component (e.g., clicking the "Publish" button), the system immediately sends a publish request for the current skill module. This request typically carries the skill module's unique identifier, version number, and the user's authentication information. Upon receiving the publish request, the server submits the generated skill module to the approval engine. In response to the approval engine's message, the server updates the skill module from draft to published status and adds it to the available skill list. Skill modules in the available skill list can be used by users to generate videos based on user input.

[0079] Users can then view the latest list of available skills by triggering an action that displays the target page (e.g., clicking the "My Skills" navigation link at the top of the interface, or accessing the "Skill Plaza" entry in the sidebar). The client responds to this action by displaying the target page. This page clearly displays all published skill modules in the form of tables, cards, or lists, naturally including the skill module that the user has just published.

[0080] The specific format of the target page can be flexibly designed according to the application scenario. For example, it could be a dedicated skills list page, centrally displaying all available skill modules and their function descriptions, version numbers, creators, and other information. Figure 4 The image shown is a skills plaza page displaying a list of skills. This list can be further divided into all skills, user-created skills, officially created skills, video generation skills, digital human skills, and so on. Figure 4 The page also provides an entry point for creating skill modules, such as an "Add Custom Skill" button. When this button is triggered, the user is redirected to the page displaying the skill module. Figure 3 The user interface shown allows users to visually arrange and generate the skill modules they want.

[0081] For example, the target page could also be a chat page with a video production AI assistant. Within this chat page, such as... Figure 5 As shown, the AI ​​assistant recommends or displays newly released skill modules to users in the form of cards. Users can select a skill module and generate the video they want by describing their needs. Furthermore, quick action buttons such as "Use Now" or "Share with Others" can be provided on the cards corresponding to the skill modules.

[0082] This design creates a complete, closed-loop workflow for users, from generating skill modules to making them usable and then guiding other users to use them, significantly lowering the barrier to skill sharing and reuse. By introducing the aforementioned second component and the target page's display mechanism, after completing workflow orchestration and generating a skill module, users no longer need to notify other users via email or offline communication; they can simply publish it with one click, and the skill module will automatically appear in a public or authorized list accessible to all users. Simultaneously, through the integration of the AI ​​assistant's conversation page, the system can proactively and intelligently recommend new skills to potential users, further promoting the widespread reuse of atomic capabilities and orchestration processes, thus making the skill module a shareable, standardized productivity tool.

[0083] Of course, there are several feasible alternatives to the above-mentioned publishing and display mechanism, which can be adjusted according to the system deployment environment and user habits. As a holistic alternative, the skill module publishing process can be integrated into the final confirmation stage of workflow orchestration, rather than existing as a separate button. For example, after the user completes the orchestration and clicks "Generate Skill," the system pops up a dialog box with two options: "Save as Draft" and "Save and Publish." When the user selects the latter, the system performs both the skill generation and the publishing request at once, thus simplifying the operation. In addition, regarding the specific form of the target page, besides a dedicated skill list page and an AI assistant conversation page, it can also be a micro-application page embedded in an instant messaging tool. Users can directly browse and access skill modules within the instant messaging tool, thus seamlessly integrating with daily workflows.

[0084] for Figure 5 The conversation page shown allows users to input materials and select skill modules. The AI ​​assistant then sends the skill module information and input materials to the execution engine, which executes the skill module and, during execution, calls the corresponding intelligent agent node to generate a video. For example, when a user... Figure 5 After uploading materials via "Upload Image / Video" on the shown conversation interface, and selecting the card corresponding to skill module A, the user sends the materials and skill module A information to the AI ​​assistant. The AI ​​assistant then sends the skill module A information and materials to the execution engine. During the video generation process, the execution engine can display information such as the execution stage, execution status, and task execution results at each stage on the conversation page, such as... Figure 6 As shown in the image. After execution, the generated video information will be returned to the user on the session interface.

[0085] In addition, the skill module generation method provided in this application also supports a dynamic workflow execution method based on user intent. Specifically, users can directly input a natural language prompt in the conversation page with the video production AI assistant, such as "Generate a 15-second short video promoting new running shoes with rhythmic music." After receiving the prompt, the server first calls a skill matcher to calculate the similarity between the feature vector of the prompt and the descriptions, metadata, or input parameter features of the published skill modules. When the similarity exceeds a preset threshold, i.e., a successful match, the skill module with the highest matching degree is directly invoked to execute the task, and the generated video result is returned to the user.

[0086] If a match fails—meaning no existing skill modules match the prompt—the user can be prompted to initiate automatic skill module orchestration. If the user responds affirmatively, the server can retrieve all registered atomic capability units from the content production framework and provide their capability descriptions, input / output schemas, and the user's prompt to a dedicated task planning AI model. This model analyzes the user's intent, automatically selects the most suitable set of atomic capability units, and designs their execution order and data dependencies, thereby dynamically generating a new ordered workflow. The server can then return this ordered workflow to the user, who can then display it on the user interface and further visualize and orchestrate it.

[0087] Figure 7 This is another method flowchart provided in an embodiment of the present application, the method being... Figure 1 The server-side execution is shown in the system architecture. For example... Figure 7 As shown, the method may include the following steps: Step 701: Provide multiple atomic capability units, each atomic capability unit corresponding to an agent node and having predefined input data structure definitions and output data structure definitions, for performing video generation-related functions.

[0088] Step 703: In response to the visual orchestration operation of the target atomic capability unit in the multiple atomic capability units, combine the target atomic capability units into an ordered workflow based on the visual orchestration operation.

[0089] Step 705: In response to the skill generation request, generate a skill module based on an ordered workflow. The skill module is an executable software component that encapsulates a topological data structure of an ordered workflow. The topological data structure includes the data dependencies and execution order between target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

[0090] As can be seen from the above process, this application predefines each corresponding intelligent agent node as an atomic capability unit with a standard input / output schema (data structure definition). Users only need to perform visual orchestration operations on these atomic capability units. This transforms manually developed, hard-coded complex logic into a structured, orchestratable workflow that can be automatically generated and executed by the system. This workflow is then encapsulated into executable software components, forming reusable skills. Users can build video generation workflows through simple visual orchestration without writing code and encapsulate them into standardized executable software components, thereby reducing reliance on users' technical knowledge and improving the efficiency of generating video ideas.

[0091] The above steps will be described in detail below with reference to embodiments. First, step 701, namely "providing multiple atomic capability units", will be described in detail with reference to embodiments.

[0092] Intelligent agent nodes can be pre-developed by researchers, with input and output schemas defined and NodeTemplateIds (node ​​identifiers) registered. Their capabilities are standardized and reusable. The pre-defined input and output schemas clearly specify the format and fields of the data the intelligent agent node needs to receive when executing a task, as well as the format and fields of the data it will output after execution. This lays the foundation for the orderly and correct transfer of data between nodes in subsequent workflows.

[0093] In this embodiment, the intelligent agent node may include, but is not limited to, input-type intelligent agents, processing-type intelligent agents, generation-type intelligent agents, understanding-type intelligent agents, and output-type intelligent agents. Specifically, processing-type intelligent agents include at least one of the following: material retrieval intelligent agent, digital human selection intelligent agent, TTS (Text To Speech) selection intelligent agent, BGM (Background Music) selection intelligent agent, and sticker selection intelligent agent. Generation-type intelligent agents may include at least one of the following: script generation intelligent agent, text-to-image intelligent agent, and text-to-video intelligent agent. Understanding-type intelligent agents include at least one of the following: image understanding intelligent agent, audio understanding intelligent agent, and text understanding intelligent agent.

[0094] In one preferred embodiment, the server can parse the functional description of the agent node from the metadata of the agent node. When multiple atomic capability units are provided, multiple atomic capability units and their corresponding functional descriptions are provided, or multiple atomic capability units and their corresponding functional tags are provided, where the functional tags are obtained based on the functional descriptions.

[0095] For example, a node named "Material Retrieval Agent" might have a description field in its metadata that reads, "Retrieves high-quality videos from first-level and second-level industries, returning only one high-quality video for analysis and video element generation." When constructing atomic capability units, the server automatically parses this description from the metadata and presents it directly to the user as the entry's explanatory text. This allows users to quickly understand the purpose of each node simply by reading its natural language description, without needing to delve into its internal implementation.

[0096] Furthermore, the server can generate a more concise function tag based on this function description through keyword extraction, semantic abstraction, or preset mapping rules. For example, the above description can be abstracted as "search for high-quality videos" or "material search". This tag can be attached to the user interface displayed on the user side in a prominent manner (such as a marker, colored icon, or prefix) so that users can quickly discover and understand the function of the atomic capability unit.

[0097] In this way, users no longer need to memorize or understand the underlying technical names of each agent node. Instead, they can intuitively understand the function of each atomic capability unit through natural language descriptions, enabling users without a technical background to get started quickly. In addition, semantic labels or descriptions facilitate users' quick location of the required atomic capability units, reducing trial and error and query time, and enabling users to more accurately and quickly assemble the required workflow.

[0098] Atomic capability units can provide a registration mechanism for developers, allowing them to add new atomic capability units. As mentioned in the preceding embodiments, multiple atomic capability units can be provided through pre-configuration or dynamic loading. In a preferred embodiment, the atomic capability unit employs an automated, event-driven registration mechanism. For example, the server continuously listens for registration events of agent nodes. When a new agent node is developed and deployed into the content production framework, a registration event is triggered. In response to this registration event, the server automatically obtains the metadata, input data structure definition, and output data structure definition of the newly registered agent node. The metadata may include basic information such as node name, node description, version number, node type (synchronous or asynchronous), and node template ID; the input data structure definition and output data structure definition follow preset standard specifications, precisely describing the data formats that the node can receive and generate. Then, the server calls a preset standardized template, which defines the basic skeleton and structure of the atomic capability unit, such as a document template containing a metadata area, an input schema area, an output schema area, and a processing logic area. The system automatically generates a complete and standardized atomic capability unit by filling in the metadata, input schema, and output schema obtained from the registration event according to the field mapping relationship of the template. For example, when a new "Text-generated Image" agent node is registered, the server automatically obtains its description "Generate Image from Text," input schema, and output schema, and fills them into a standardized template to generate an atomic capability unit named "Text-generated Image_Skill". Through this automated generation mechanism, the creation of atomic capability units is tightly bound to the lifecycle of the underlying agent node, achieving the automated effect of capability availability upon node registration.

[0099] This approach eliminates the need for manual development, testing, and maintenance of atomic capability units for each underlying node, significantly reducing maintenance costs and redundant manpower. Secondly, this event-driven, automated generation model greatly enhances the system's scalability, enabling automatic and batch standardized encapsulation, thereby accelerating the construction speed of the entire atomic capability system.

[0100] Of course, there are several feasible alternatives to the above-mentioned automated generation method. For example, the generation of atomic capability units can be achieved without relying on real-time registration events, but rather through timed polling or batch scanning.

[0101] The following describes step 705, namely "in response to a skill generation request, generate a skill module based on an ordered workflow," in detail with reference to an embodiment.

[0102] Upon receiving a skill generation request, the server executes a packaging and encapsulation process. First, it serializes and structures the received ordered workflow definition, extracting its core component: the topology data structure. This topology data structure is not simply a list of nodes or edges, but contains all the information needed by the execution engine: a unique identifier for each node, dependencies between nodes, parameters required for node execution and their values, and the global initial context definition of the workflow. This structured, machine-readable topology data structure is then embedded into a pre-defined software component framework, generating a skill module that can be independently deployed and invoked.

[0103] The skill module is essentially an executable software component. It no longer relies on raw, graphical canvas information, but instead solidifies the workflow logic programmatically. When this video generation task needs to be executed, simply calling the skill module's standardized interface (e.g., passing in a JSON object containing user scripts, images, etc.) allows the skill module to automatically parse the execution order based on its internally encapsulated topological data structure and call the corresponding underlying intelligent agent nodes accordingly, ultimately generating a complete video. For example, a workflow previously built by dragging and dropping "Script Generation," "Text-to-Image Generation," "Image-to-Video Generation," and "Video Rendering" nodes will be parsed by the execution engine after the skill module is generated. The system will first call the script generation node, passing its output to the text-to-image generation node, then the image-to-video generation node, and finally the rendering node. All steps are executed automatically according to the definition, requiring no manual intervention. Through this "orchestrated once, reused continuously" mechanism, this application transforms users' video ideas into standardized software assets that can be directly called by machines, greatly accelerating the efficiency of video idea generation.

[0104] To further ensure the correctness and predictability of user-orchestrated workflows at the execution level, users can trigger trial runs of ordered workflows. In response to a trial run request for an ordered workflow, the ordered workflow is submitted to the workflow execution engine to generate trial run results, which are then returned. The trial run results include the execution status and output data of each atomic capability unit in the ordered workflow.

[0105] To improve the complete lifecycle management of skill modules from generation to deployment, users can quickly publish skill modules through the user interface. In response to a user's publishing request, the generated skill module is submitted to the publishing approval engine. Upon receiving an approval message from the engine, the skill module is updated from draft to published and added to the available skill list. Skill modules in the available skill list can be used by users to generate videos based on user input.

[0106] The list of available skills can be displayed on pages such as the skills list page and the conversation page between the user and the video production AI assistant. For details on the implementation process described above, please refer to [link to relevant documentation]. Figure 2 The relevant details in the illustrated embodiments will not be repeated here.

[0107] By introducing the aforementioned trial run and release approval processes, the robustness and controllability of skill module generation are further enhanced. The trial run function allows users to identify and correct problems at a low cost during the ideation stage, preventing erroneous logic from being carried over into subsequent generation and release stages. The release approval function adds a quality control barrier before the skill module goes live, effectively preventing low-quality or security-risk skills from impacting online business. Together, these two processes construct a complete and reliable production pipeline from idea verification to quality review and official launch, significantly reducing system maintenance risks and business incident rates, and improving overall delivery quality.

[0108] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0109] Figure 8 This is a schematic block diagram of a skill module generation apparatus for video generation provided in an embodiment of this application. The apparatus is disposed in... Figure 1 The user end in the illustrated architecture. (For example...) Figure 8As shown, the device 800 includes: an interface display unit 801, an operation monitoring unit 802, and a request sending unit 803. The main functions of each component are as follows: The interface display unit 801 is configured to display a user interface, which includes a list area and a canvas area. The list area displays multiple atomic capability units, each of which corresponds to an agent node and has predefined input data structure definitions and output data structure definitions, used to perform video generation-related functions.

[0110] Operation listening unit 802 is configured to listen for operations on the user interface.

[0111] The interface display unit 801 is also configured to respond to the operation listening unit 802 listening to the visualization orchestration operation of the target atomic capability unit among multiple atomic capability units, and to display the orchestration result of the target atomic capability unit in the canvas area. The orchestration result is used to combine the target atomic capability units into an ordered workflow.

[0112] The request sending unit 803 is configured to send a skill generation request in response to the operation listening unit 802 detecting an operation that triggers the generation of a skill. The skill module is an executable software component that encapsulates a topological data structure of an ordered workflow. The topological data structure includes the data dependencies and execution order between target atomic capability units. The skill module is used to call the corresponding intelligent agent node to generate video based on the data dependencies and execution order.

[0113] As one possible implementation method, the interface display unit 801 can be specifically configured to display entries with multiple atomic capability units in the list area. The entries include the functional description or functional tag of the corresponding agent node, wherein the functional description is obtained from the metadata of the agent node, and the functional tag is obtained based on the functional description.

[0114] As one possible implementation, the interface display unit 801 can be specifically configured to: in response to an operation of pressing and holding a target atomic capability unit in the list area, display a visual element of the target atomic capability unit near the pressed position; in response to a continued pressing operation and a drag operation from the list area to the canvas area, set the visual element at the end position of the drag operation on the canvas area; in response to an operation of connecting two visual elements in the canvas area, set a directed edge between the two visual elements in the canvas area, the directed edge being used to generate data dependencies and execution order between the target atomic capability units.

[0115] As one possible implementation method, when the interface display unit 801 sets a directed edge between two visual elements in the canvas area, it can be specifically configured as follows: if the two visual elements satisfy the dependency constraint, then set a directed edge between the two visual elements in the canvas area; if the two visual elements do not satisfy the dependency constraint, then display a prompt message on the user interface, the prompt message being used to indicate at least one of the following: the directed edge setting failed and the reason for the failure.

[0116] Furthermore, the user interface also includes a control area; the request sending unit 803 can also be configured to: in response to an operation that selects a target visual element in the canvas area, request the configurable parameters of the target visual element. The interface display unit 801 can also be configured to: display the configurable parameters of the target visual element in the control area. The request sending unit 803 can also be configured to: in response to a modification operation of the configurable parameters, send an update request to update the value of the configurable parameters of the agent node corresponding to the target visual element.

[0117] Furthermore, the user interface also includes a control area that displays the first component that triggered the trial run workflow. The request sending unit 803 can also be configured to send a trial run request for the ordered workflow in response to the operation that triggered the first component. The interface display unit 801 can also be configured to acquire and display the trial run results of the ordered workflow on the user interface, the trial run results including the execution status and output data of each atomic capability unit in the ordered workflow.

[0118] Furthermore, the user interface also displays a second component that triggers skill release; the request sending unit 803 can also be configured to send a release request for the skill module in response to the operation that triggers the second component. The interface display unit 801 can also be configured to display a target page in response to the operation that triggers the display of a target page, the target page displaying a list of available skills, the list of available skills containing the released skill modules, wherein the target page includes a skill list page or a conversation page with the video production AI assistant.

[0119] Figure 9 This is a schematic block diagram of another skill module generation apparatus for video generation provided in an embodiment of this application. The apparatus is disposed in... Figure 1 The server side in the illustrated architecture. For example... Figure 9 As shown, the device 900 includes: an atom providing unit 901, a workflow assembly unit 902, and a skill generation unit 903. It may also include an atom registration unit 904, a trial operation unit 905, and a skill release unit 906. The main functions of each component are as follows: Atom providing unit 901 is configured to provide multiple atomic capability units, each atomic capability unit corresponding to an agent node and having predefined input data structure definitions and output data structure definitions, for performing video generation-related functions.

[0120] Workflow composition unit 902 is configured to respond to a visual orchestration operation on a target atomic capability unit among multiple atomic capability units, and to compose the target atomic capability units into an ordered workflow based on the visual orchestration operation. The ordered workflow defines the data dependencies and execution order between the target atomic capability units.

[0121] The skill generation unit 903 is configured to generate a skill module based on an ordered workflow in response to an operation that triggers the generation of a skill. The skill module is an executable software component that encapsulates a topological data structure of an ordered workflow and is used to call the corresponding agent node to generate video based on data dependencies and execution order.

[0122] Furthermore, the atomic registration unit 904 is configured to respond to the registration event of the agent node, obtain the agent node's metadata, input data structure definition, and output data structure definition; and call a preset standardized template to generate the atomic capability unit corresponding to the agent node based on the agent node's metadata, input data structure definition, and output data structure definition.

[0123] Furthermore, the atomic providing unit 901 is also configured to: parse the functional description of the agent node from the agent node's metadata. When providing multiple atomic capability units, it can be configured to: provide multiple atomic capability units and their corresponding functional descriptions, or provide multiple atomic capability units and their corresponding functional tags, where the functional tags are obtained based on the functional descriptions.

[0124] Furthermore, the trial run unit 905 can be configured to: in response to a trial run request for an ordered workflow, submit the ordered workflow to the workflow execution engine to generate trial run results, and return the trial run results, which include the execution status and output data of each atomic capability unit in the ordered workflow.

[0125] Furthermore, the skill publishing unit 906 can be configured to: in response to a publishing request for a skill module, submit the generated skill module to the publishing approval engine; in response to the publishing approval engine returning an approval message, update the skill module from draft status to published status and add it to the available skill list; the skill modules in the available skill list can be called by users to generate videos based on user input.

[0126] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The system and device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0128] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0129] And an electronic device, comprising: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0130] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0131] in, Figure 10An exemplary architecture of an electronic device is shown, which may include a processor 1010, a video display adapter 1011, a disk drive 1012, an input / output interface 1013, a network interface 1014, and a memory 1020. The processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020 can communicate with each other via a communication bus 1030.

[0132] The processor 1010 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0133] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system 1021 for controlling the operation of the electronic device 1000, and the basic input / output system (BIOS) 1022 for controlling the low-level operations of the electronic device 1000. Additionally, it can store a web browser 1023, a data storage management system 1024, and skill module generation devices 800 / 900, etc. The aforementioned skill module generation devices 800 / 900 can be the application programs that specifically implement the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 1020 and executed by the processor 1010.

[0134] Input / output interface 1013 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0135] The network interface 1014 is used to connect the communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0136] Bus 1030 includes a pathway for transmitting information between various components of the device, such as processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020.

[0137] It should be noted that although the above-described device only shows the processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, memory 1020, bus 1030, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0138] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0139] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating skill modules for video generation, characterized in that, The method includes: The user interface is displayed, which includes a list area and a canvas area. The list area displays multiple atomic capability units, each of which corresponds to an agent node and has a predefined input data structure definition and output data structure definition, used to perform video generation-related functions. In response to the visualization and arrangement operation of the target atomic capability unit among the plurality of atomic capability units, the arrangement result of the target atomic capability unit is displayed in the canvas area, and the arrangement result is used to combine the target atomic capability units into an ordered workflow; In response to the operation that triggers skill generation, a skill generation request is sent to request the generation of a skill module based on the ordered workflow. The skill module is an executable software component that encapsulates the topology data structure of the ordered workflow. The topology data structure includes the data dependencies and execution order between the target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

2. The method according to claim 1, characterized in that, The list area displays multiple atomic capability units, including: entries displaying multiple atomic capability units in the list area, each entry including a functional description or functional tag of the corresponding agent node, wherein the functional description is parsed from the metadata of the agent node, and the functional tag is obtained based on the functional description.

3. The method according to claim 1, characterized in that, The visual orchestration operation includes: In response to an operation of pressing and holding a target atomic capability unit in the list area, a visual element of the target atomic capability unit is displayed near the pressed and held position; In response to a follow-up press and drag operation from the list area to the canvas area, the visual element is set at the end position of the drag operation on the canvas area. In response to an operation that connects two visual elements in the canvas area, a directed edge is set between the two visual elements in the canvas area, the directed edge being used to generate data dependencies and execution order between target atomic capability units.

4. The method according to claim 3, characterized in that, Setting a directed edge between the two visual elements in the canvas area includes: If the two visualization elements satisfy a dependency constraint, then a directed edge is set between the two visualization elements in the canvas area; If the two visual elements do not satisfy the dependency constraint, a prompt message is displayed on the user interface to indicate at least one of the following: the directed edge setting failed and the reason for the failure.

5. The method according to claim 3, characterized in that, The user interface also includes a control area; In response to the selection of a target visual element in the canvas area, request and display configurable parameters of the target visual element in the control area; In response to the modification operation of the configurable parameters, an update request is sent to update the value of the configurable parameters of the agent node corresponding to the target visualization element.

6. The method according to any one of claims 1 to 5, characterized in that, The user interface also includes a control area that displays a first component for triggering the trial run workflow; the method further includes: In response to the operation that triggers the first component, a trial run request is sent for the ordered workflow; The trial run results of the ordered workflow are obtained and displayed on the user interface. The trial run results include the execution status and output data of each atomic capability unit in the ordered workflow.

7. The method according to any one of claims 1 to 5, characterized in that, The user interface also displays a second component that triggers the skill release; the method further includes: In response to the operation that triggers the second component, a release request for the skill module is sent; In response to an operation that triggers the display of a target page, the target page is displayed, which displays a list of available skills, including the published skill modules, wherein the target page includes a skill list page or a conversation page with a video production AI assistant.

8. A method for generating skill modules for video generation, characterized in that, The method includes: Multiple atomic capability units are provided, each of which corresponds to an agent node and has a predefined input data structure definition and output data structure definition, for performing video generation-related functions; In response to a visual orchestration operation on a target atomic capability unit among the plurality of atomic capability units, the target atomic capability units are combined into an ordered workflow based on the visual orchestration operation; In response to a skill generation request, a skill module is generated based on the ordered workflow. The skill module is an executable software component that encapsulates the topology data structure of the ordered workflow. The topology data structure includes the data dependencies and execution order between the target atomic capability units. The skill module is used to call the corresponding agent node to generate video according to the data dependencies and execution order.

9. The method according to claim 8, characterized in that, The atomic capability units are generated in the following manner: In response to the registration event of the agent node, obtain the metadata, input data structure definition and output data structure definition of the agent node; By invoking a preset standardized template, atomic capability units corresponding to the agent node are generated based on the metadata, input data structure definition, and output data structure definition of the agent node.

10. The method according to claim 9, characterized in that, The method further includes: parsing the functional description of the agent node from the metadata of the agent node; Providing multiple atomic capability units includes: providing the multiple atomic capability units and their corresponding functional descriptions, or providing the multiple atomic capability units and their corresponding functional labels, wherein the functional labels are obtained based on the functional descriptions.

11. The method according to any one of claims 8 to 10, characterized in that, The method further includes at least one of the following: In response to a trial run request for the ordered workflow, the ordered workflow is submitted to the workflow execution engine to generate trial run results, and the trial run results are returned, including the execution status and output data of each atomic capability unit in the ordered workflow; In response to a release request for the skill module, the generated skill module is submitted to the release approval engine. In response to the release approval engine returning a message indicating approval, the skill module is updated from draft to released and added to the available skill list. The skill modules in the available skill list can be called by the user to generate a video based on the user's input.

12. A skill module generation device for video generation, characterized in that, The device includes: The interface display unit is configured to display a user interface, which includes a list area and a canvas area. The list area displays multiple atomic capability units, each of which corresponds to an agent node and has a predefined input data structure definition and an output data structure definition, used to perform video generation-related functions. An operation monitoring unit is configured to monitor operations on the user interface. The interface display unit is further configured to, in response to the operation monitoring unit detecting a visual arrangement operation on a target atomic capability unit among the plurality of atomic capability units, display the arrangement result of the target atomic capability unit in the canvas area, wherein the arrangement result is used to combine the target atomic capability units into an ordered workflow; The request sending unit is configured to send a skill generation request in response to the operation listening unit detecting an operation that triggers skill generation. The skill module is an executable software component that encapsulates the topology data structure of the ordered workflow. The topology data structure includes the data dependencies and execution order between the target atomic capability units. The skill module is used to call the corresponding agent node to generate video based on the data dependencies and execution order.

13. A skill module generation device for video generation, characterized in that, The device includes: An atom providing unit is configured to provide a plurality of atomic capability units, each of which corresponds to an agent node and has a predefined input data structure definition and an output data structure definition, for performing video generation-related functions; A workflow composition unit is configured to respond to a visual orchestration operation on a target atomic capability unit among the plurality of atomic capability units, and to combine the target atomic capability units into an ordered workflow based on the visual orchestration operation, wherein the ordered workflow defines the data dependencies and execution order between the target atomic capability units; The skill generation unit is configured to generate a skill module based on the ordered workflow in response to an operation that triggers the generation of a skill. The skill module is an executable software component that encapsulates the topological data structure of the ordered workflow and is used to call the corresponding agent node to generate video according to the data dependencies and execution order.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 11.

15. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 11.

16. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1 to 11.

Images