Multi-screen visualization collaboration method and system based on agent and related device

Abstract

The application discloses a multi-screen visualization collaboration method and system based on agents, and related devices, relates to the technical field of human-computer interaction, and the method is applied to a portable computing device and comprises the following steps: receiving and analyzing a task instruction input by a user to generate a standardized task request; constructing a plurality of agents in a local mode to form an agent cluster, generating a task directed acyclic graph based on the task request; binding the plurality of agents to different display terminals one by one, and when detecting that the load of a first agent is too high, migrating part of a task of the first agent to a second agent with lower load, synchronously updating the display content of the corresponding display terminals; triggering human-computer interaction when a preset key decision node is reached; and after the task is completed, auditing and finally processing and storing the results. In this way, the application realizes the visualization presentation of the multi-agent collaborative computing process, load balancing and human-computer interaction intervention at key links, and improves the reliability and efficiency of the agent in processing complex tasks.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a multi-screen visual collaboration method, system and related device based on intelligent agents. Background Technology

[0002] Currently, in professional fields such as multimedia content production, data analysis, and software development, complex tasks are typically broken down into processes executed sequentially or in parallel by multiple specialized software programs or modules. To improve efficiency, professionals often use multiple monitors to expand their workspace.

[0003] Existing multi-screen collaboration solutions mainly fall into two categories: one is network-based distributed computing systems, which distribute tasks to multiple computing nodes. However, these systems are complex to interact with, have high latency, and require a stable network environment, making them unsuitable for mobile or on-site creation scenarios that require rapid response and privacy protection. The other category involves connecting multiple monitors to a single workstation, requiring users to manually switch application windows between different screens to obtain real-time status updates for different stages. However, this approach lacks automation and intelligent scheduling; the correspondence between task status and display terminals is static and manually maintained, failing to dynamically reflect the logical dependencies between multiple task flows and the real-time load of the devices. This results in cumbersome operation, distraction, and low overall collaboration efficiency.

[0004] Therefore, how to achieve visual collaborative interaction between task execution processes and multiple display terminals in the local computing environment of the device has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a multi-screen visual collaboration method, system, and related devices based on intelligent agents. The method assigns corresponding display terminals to multiple intelligent agents, invokes the agents to collaboratively execute a target task, triggers human-computer interaction on a designated main interactive screen among the multiple display terminals, and executes the task target based on the results of the human-computer interaction until completion. The specific technical solution is as follows:

[0006] Firstly, a multi-screen visual collaboration method based on intelligent agents is provided, applied to portable computing devices. The method includes: receiving and parsing user-input task instructions to generate standardized task requests; responding to the standardized task requests, deploying at least two intelligent agents constituting an intelligent agent cluster locally using a containerized engine, generating a directed acyclic graph (DAG) of tasks based on the standardized task requests, and mounting external storage devices to initialize the project workspace; based on the DAG and a preset mapping strategy, mapping and binding each intelligent agent in the intelligent agent cluster to at least two display terminals connected to the portable computing device, thereby displaying the subtask execution process of each intelligent agent on their respective bound display terminals; and enabling collaboration of the intelligent agent cluster based on the task DAG. When executing subtasks, if the load of the first agent in the agent cluster is greater than or equal to a preset threshold, at least one subtask undertaken by the first agent is migrated to the second agent in the agent cluster whose load is less than the preset threshold. The display content in the display terminals bound to the first and second agents is updated synchronously. When the process in the directed acyclic graph of tasks reaches a preset key decision node, the subsequent process is paused, and a human-computer interaction request is triggered in the display terminal designated as the main interaction screen. Based on the user input of the human-computer interaction, the task parameters are updated, and the execution process in the directed acyclic graph of tasks is resumed. After the execution of the directed acyclic graph of tasks is completed, the output results are reviewed and finally processed, and the final results are stored in the project workspace and a delivery notification is sent.

[0007] In conjunction with the first aspect, receiving and parsing user-inputted task instructions specifically includes: receiving user-inputted task instructions via voice input device; performing intent recognition on the task instructions using a locally deployed natural language processing model; and generating standardized task requests based on the intent recognition results.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the intelligent agent cluster includes a global management intelligent agent that generates a directed acyclic graph of tasks based on standardized task requests and mounts an external storage device to initialize the project workspace. Specifically, the global management intelligent agent performs the following actions: the global management intelligent agent parses the standardized task requests, plans the task flow to generate the directed acyclic graph of tasks, and creates a corresponding project workspace for the directed acyclic graph of tasks on the external storage device.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, at least two agents constituting an agent cluster are deployed locally by invoking a containerization engine, specifically including: the global management agent deploying at least one project processing container through the containerization engine, the project processing container being used to host each agent executing subtasks.

[0010] In conjunction with the first aspect, in some embodiments of the first aspect, the intelligent agent cluster also includes a material organization intelligent agent, an editing intelligent agent, and a special effects compositing intelligent agent. Each intelligent agent in the intelligent agent cluster is mapped one-to-one and bound to at least two display terminals connected to a portable computing device. Specifically, this includes binding the material organization intelligent agent to a first display terminal, the editing intelligent agent to a second display terminal, the special effects compositing intelligent agent to a third display terminal, and binding the global control intelligent agent to the main interactive screen.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the subtask execution process of each intelligent agent is displayed on the respective bound display terminals, specifically including: in a video processing scenario, a material library browsing and filtering interface is displayed on the first display terminal, a video timeline editing interface is displayed on the second display terminal, a special effects parameter adjustment interface is displayed on the third display terminal, and the entire process status of the directed acyclic graph of the task, the progress of each intelligent agent, and the system resource usage are comprehensively displayed on the main interactive screen.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the intelligent agent cluster collaboratively executes sub-tasks based on a directed acyclic graph (DAG), specifically including: the global management agent sequentially scheduling the sub-tasks corresponding to each node in the DAG according to the topological order of the DAG; when the material organization agent, editing agent, or special effects compositing agent completes the execution of the sub-task of the upstream node in the DAG, it obtains the corresponding output data and status identifier; the global management agent, based on the dependency relationship defined by the edges in the DAG, uses the output data as input to instruct the material organization agent, editing agent, or special effects compositing agent corresponding to the downstream node to execute the corresponding sub-task, until the sub-tasks corresponding to each node in the DAG are completed.

[0013] In conjunction with the first aspect, in some embodiments of the first aspect, the first intelligent agent and the second intelligent agent are any two different intelligent agents among the global management intelligent agent, the material organization intelligent agent, the editing intelligent agent, and the special effects compositing intelligent agent. Updating the display content in the display terminals bound to the first intelligent agent and the second intelligent agent specifically includes: synchronously mapping the interactive interface and real-time status related to the sub-task to be migrated in the first intelligent agent from the display terminal bound to the first intelligent agent to the display terminal bound to the second intelligent agent for display; displaying task migration prompt information on the display terminal bound to the first intelligent agent, and displaying the execution status window corresponding to the received sub-task on the display terminal bound to the second intelligent agent.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the key decision nodes include a video main segment automatic splicing completion node and a video effect style selection node to be added. A human-computer interaction request is triggered on the display terminal designated as the main interactive screen. Specifically, when the key decision node is the video main segment automatic splicing completion node, a splicing effect preview and confirmation controls are displayed on the main interactive screen; when the key decision node is the video effect style selection node to be added, a style candidate list and selection controls are displayed on the main interactive screen.

[0015] It should be noted that, in the absence of conflict, the features in the various embodiments of the first aspect can be combined with each other, and any combination of features in different embodiments is also within the protection scope of this application. That is to say, the various embodiments described above can also be arbitrarily combined according to actual needs.

[0016] In a second aspect, a multi-screen visual collaboration system based on intelligent agents is provided to implement the method as described in any of the first aspects. The system includes: a portable computing device as the core computing and scheduling unit; an external storage device communicatively connected to the portable computing device for providing project workspace and material storage; and at least two display terminals connected to the portable computing device for visually presenting the display interfaces and system status of different intelligent agents. The portable computing device is configured with functional modules for executing the method steps as described in any of the first aspects.

[0017] In the embodiments of this application, the method provided by this application has the following beneficial effects:

[0018] 1. Achieve localized collaboration among multiple intelligent agents. The core computing and collaboration logic of this application is deployed in a containerized environment on a portable computing device (such as a high-performance laptop, a microcomputer with pre-installed artificial intelligence (AI Mini PC), etc.). All data processing is completed locally on the computing device without relying on the cloud or remote servers, which ensures data security, reduces network latency, and makes the workstation portable.

[0019] 2. Achieve dynamic visualization mapping between task flow and multiple screens. This application's solution visualizes the execution status of each agent in the task flow on the display terminal, intuitively demonstrating the internal workflow of the agent. Furthermore, this application's solution can dynamically adjust the displayed content based on task load and dependencies, intuitively showing the progress, bottlenecks, and interrelationships of the entire task flow.

[0020] 3. Intelligent Decision-Making and Human-Computer Interaction. This application's solution can automatically trigger the corresponding interactive interface on the main interactive screen when the task flow reaches key nodes requiring manual decision-making, parameter fine-tuning, or result review. This achieves automatic execution of the solution and seamless integration with the intelligent agent, reducing unnecessary waiting time and switching operations for users, and optimizing the human-computer interaction experience.

[0021] 4. Achieve coordinated resource scheduling. This application's solution, combined with visual feedback, can analyze and calculate resource utilization efficiency and allocate resources rationally, ensuring that critical tasks receive sufficient resources and improving overall task execution efficiency.

[0022] 5. Achieving Synergistic Technical Effects. This application's solution combines containerized agent scheduling, task flow management based on directed acyclic graphs (DAGs), and multi-screen visualization mapping to enable real-time transparency and intervention in the automated task execution process. This combination of technical features helps to ensure local data processing security while achieving intuitive detection and flexible scheduling of complex task flows. Its overall effect surpasses the simple summation of the individual technical features. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a system framework provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of a portable computing device provided in an embodiment of this application;

[0026] Figure 3 This is a flowchart of a multi-screen visual collaboration method based on intelligent agents provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram illustrating load migration between intelligent agents and synchronous content updates provided in an embodiment of this application;

[0028] Figure 5 This is a flowchart of a multi-screen visual collaboration method tailored to a specific scenario, provided in an embodiment of this application.

[0029] Figure 6 This is an interactive timing diagram of a multi-screen visual collaboration method combined with a specific scenario provided in an embodiment of this application;

[0030] Figure 7 This is a schematic diagram illustrating a scenario for rapid short video content generation provided in an embodiment of this application;

[0031] Figure 8 This is a schematic diagram of the module structure of a multi-screen visual collaboration system based on intelligent agents provided in an embodiment of this application;

[0032] Figure 9 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application;

[0033] Figure 10 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0036] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0037] The following describes the agent-based multi-screen visual collaboration method, system, and related apparatus provided in this application through three embodiments. Embodiment 1 describes the system framework of the agent-based multi-screen visual collaboration method, Embodiment 2 describes the process of implementing the agent-based multi-screen visual collaboration method, and Embodiment 3 describes the system module structure, computer device hardware structure, and computer-readable storage medium for executing the agent-based multi-screen visual collaboration method.

[0038] Example 1:

[0039] Figure 1 This is a schematic diagram of a system framework provided in an embodiment of this application. For example... Figure 1 As shown, the system framework 100 can be divided into two main parts: hardware architecture 110 and software architecture 120. The two work together to achieve localized and highly observable intelligent task processing.

[0040] In this embodiment, the hardware architecture 110 constitutes the hardware foundation and computing carrier of the system, and mainly includes the following key components: portable computing device 111, multiple display terminals 112, network-attached storage device 113, data input device 114, and local network device 115.

[0041] In this embodiment, the portable computing device 111 is the core computing unit of the system framework 100, used to carry and run the entire software architecture. The portable computing device 111 is preferably a microcomputer (AI Mini PC) pre-configured with artificial intelligence, specifically equipped with at least an 8-core Central Processing Unit (CPU), a Graphics Processing Unit (GPU) with dedicated artificial intelligence (AI) computing power to support AI computation and inference, and integrating multiple interfaces (e.g., a High Definition Multimedia Interface (HDMI) interface and two Universal Serial Bus (USB) interfaces) to support simultaneous driving of four display outputs. This configuration supports the computing resources required for the localized deployment and operation of multiple intelligent agent containers in this embodiment. A more detailed description of the portable computing device 111 can be found in [reference needed]. Figure 2 This will not be elaborated upon here.

[0042] In this embodiment, multiple display terminals 112 are multiple physically independent displays connected to the portable computing device 111 via a video output interface, jointly realizing human-computer interaction and status detection of the system. At least one is designated as the main interaction screen, which is the highest priority interaction channel in the system. It is mainly used to: display the full task flow status view provided by the global management agent, the overall load dashboard of the agent cluster, etc.; when the task execution reaches a critical decision node, the main interaction screen will automatically pop up and focus on displaying a specific interactive interface (such as a preview confirmation dialog box, parameter adjustment panel), used to receive user input commands and realize the pause and controlled resumption of the process. The remaining display terminals among the multiple display terminals 112 serve as extended display terminals, used to dynamically map and display the exclusive working interface and real-time status of each agent. During task execution, each extended display terminal is bound one-to-one with a specific task agent (such as a material organization agent, editing agent) through a dynamic mapping module. This display terminal will display the agent's execution interface, operation log, progress bar, intermediate result preview, and resource usage in real time. This binding relationship can also be dynamically adjusted based on the structure of the task directed acyclic graph and the load migration strategy.

[0043] In this embodiment, the network-attached storage device 113 and the portable computing device 111 are connected via a high-speed bus or a direct-connect external storage array to serve as the system's project workspace. For example, network-attached storage (NAS) can be connected to the portable computing device 111 via a high-speed direct-connect interface (such as a Thunderbolt 4 interface or a 10 Gigabit Ethernet interface) to provide the agent with high-bandwidth, low-latency access services for project materials, model weights, and result files, ensuring that data is processed entirely locally. Specifically, the functions of the network-attached storage device 113 include:

[0044] 1. Store the original materials required for task execution (such as video and audio files), intermediate data generated during processing, AI model weight files, and final output results;

[0045] 2. Provide high-bandwidth, low-latency data read and write services for various intelligent agents running on portable computing devices, ensuring that large-capacity data such as video streams can be exchanged smoothly between intelligent agents. This is the physical basis for achieving efficient localized processing and avoiding cloud transmission bottlenecks.

[0046] 3. In collaboration with data and state management services, it saves data such as directed acyclic graphs of tasks, snapshots of agent states, and user interaction history, and supports real-time pause of tasks, cross-session resumption, and historical rollback.

[0047] In this embodiment, the data input device 114 includes peripherals such as a keyboard, mouse, graphics tablet, and touchpad connected to the portable computing device 111. These provide input means for user interaction across the entire system interface. When making key decisions on the main interactive screen (such as clicking to confirm, dragging parameter sliders, or annotating content), the data input device 114 can provide precise operational feedback. Furthermore, the data input device 114 can also be used to directly operate the interface of the bound intelligent agent on an extended display terminal (such as adjusting the editing timeline).

[0048] In this embodiment, the local network device 115 can be a wireless network card / wired network port built into the portable computing device 111, or it can be a network switch connected to the portable computing device 111. Its main function is:

[0049] 1. Before the task begins, download or transfer the original materials from the Internet, a remote server, or other devices to the project workspace of the network-attached storage device 113;

[0050] 2. During system operation, a secure network connection is provided, allowing authorized users to perform remote desktop access, status monitoring, or log debugging without interfering with the core data processing pipeline of the local intelligent agent cluster, thus ensuring the isolation and security of the core processes.

[0051] In this embodiment, the software architecture 120 is deployed on the operating system of the portable computing device 111, including a containerization engine and runtime environment 121 and an intelligent agent cluster 122. Its core is the intelligent agent cluster 122, especially the global management intelligent agent within the cluster, which coordinates with other functional modules to jointly execute a multi-screen visual collaboration method based on intelligent agents. The software architecture 120 mainly includes the following functional modules:

[0052] In this embodiment, the containerization engine and runtime environment 121 are used to provide a containerized runtime environment for the intelligent agent cluster 122 to call, so as to deploy and manage the lifecycle and resource isolation of each task execution intelligent agent (such as material organization intelligent agent, editing intelligent agent, special effects compositing intelligent agent, etc.) constituting the intelligent agent cluster 122.

[0053] In some embodiments, the containerization engine and runtime environment 121 can be implemented using, for example, the application container engine Docker or similar technologies. Communication within the agent cluster 122, such as the global management of agents scheduling specialized agents for material organization, editing, etc., can be achieved through application programming interface (API) calls, message queues, or shared networks and storage volumes defined between containers.

[0054] In this embodiment, the agent cluster 122 serves as the system's control center, specifically responsible for: parsing user task instructions based on the global management agent within it, generating a directed acyclic graph (DAG); creating and managing project workspaces on the network-attached storage device 113; scheduling agents to collaboratively execute sub-tasks according to the topological order of the DAG; real-time monitoring of the load status of each agent, and when the first agent's load is detected to be too high, deciding and migrating the sub-task to the second agent; and triggering a human-computer interaction request on the main interactive screen when the process reaches a critical decision node, and updating parameters based on user input. The first agent and the second agent are any two different agents within the agent cluster 122.

[0055] In this embodiment, the dynamic mapping module 123 is responsible for driving and displaying the subtask execution process and status of each intelligent agent in real time on the specific display terminal bound to it, based on the scheduling and status information of the global management intelligent agent, and synchronously updating the interface content of the relevant display terminal when the task is migrated.

[0056] Specifically, the dynamic mapping module 123 can use the multi-monitor management interface provided by the operating system, combined with the graphical user interface framework, to render and display the status data (such as progress, log stream, and preview images) output by each agent to the specific display terminal bound to each agent. When a task migration occurs, the dynamic mapping module 123 can close the relevant task windows on the original screen and create a new window on the screen bound to the target agent, while updating the input source of the task status data.

[0057] In this embodiment, the resource detection and subtask migration module 124 is used to continuously collect the resource utilization rate of each agent (such as central processing unit (CPU), graphics processing unit (GPU), memory, etc.), to provide data support for the load judgment and migration decision of the global management agent, and to assist in the saving, transmission and recovery of the execution task context data.

[0058] Specifically, when making a migration decision, the resource detection and subtask migration module 124 saves the execution context (including memory state, temporary file handles, etc.) of the subtask to be migrated in the first agent through the containerization engine interface, and synchronizes the context data to the second agent through shared storage or network transmission. The second agent loads the context and continues to execute, thereby ensuring the continuity and data consistency before and after the task migration.

[0059] In this embodiment, the data and state management service 125 maintains data such as the directed acyclic graph of tasks, agent states, mapping relationships, and user interaction history. It supports pausing, resuming, and retracing the task flow and manages data synchronization between the portable computing device 111 and the network-attached storage device 113. For example, the data and state management service 125 can instruct the corresponding agent to read raw task data from the network-attached storage device 113 and store the intermediate and final result data generated by the agent back into the network-attached storage device 113.

[0060] Through the deep integration of the aforementioned hardware and software architecture, system framework 100 can achieve multi-screen visual collaboration and real-time intervention in the multi-agent collaborative computing process on a single portable device. It is understood that the functional division between the modules illustrated in the embodiments of this application is merely illustrative and does not constitute a limitation on the functionality of system framework 100. In other embodiments of this application, system framework 100 may also employ different modules or combinations of multiple modules to implement the functions within system framework 100.

[0061] Figure 2 This is a schematic diagram of a portable computing device provided in an embodiment of this application. Figure 2 As shown, the portable computing device 111 has a rectangular plastic casing and multiple interfaces.

[0062] In some embodiments, the portable computing device 111 may be configured with the following hardware parameters: the entire casing is made of plastic (dimensions, for example: 147mm long, 147mm wide, and 61mm high), supports wireless communication via Wi-Fi and Bluetooth, and includes: two USB ports, two Type-C ports for video and data transmission, one HDMI port and one DisplayPort (DP) port for audio and video data transmission, an RJ45 connector for network transmission and two 2.5G Ethernet ports, a DC power input port, and two audio input / output ports.

[0063] In some implementations, the portable computing device 111 operates normally at an input voltage of 19V and an input current of 15.79A, with an operating temperature between 0 and 40°C, a storage temperature between -20 and 60°C, and can operate normally in an environment with 90% humidity at 40°C.

[0064] In some implementations, the portable computing device 111 includes an 8-core CPU, a GPU supporting AI computation and inference, 64GB of RAM, and a 2TB solid-state drive (SSD), and supports running different operating systems (OS), such as Windows and Linux. The CPU, GPU, RAM, and SSD integrated into the portable computing device 111 can also be adjusted according to actual needs.

[0065] Example 2:

[0066] Figure 3 This is a flowchart of a multi-screen visual collaboration method based on an intelligent agent, provided in an embodiment of this application, which is applied to, for example... Figure 1 The system framework 100 shown specifically includes:

[0067] S101. Receive and parse the task instructions input by the user, and generate a standardized task request.

[0068] In this embodiment, the system can receive user voice commands via a voice input device and perform intent recognition using a locally deployed natural language processing model. Based on the recognition results, it generates structured, standardized task requests. Specifically, the system can receive user voice input requesting the creation of a promotional video via a local model and convert it into text. Then, based on a predefined domain knowledge base, it decomposes the target task into a series of sequential or parallel sub-tasks (e.g., sequentially performing material collection, rough editing, adding special effects, rendering output, etc.).

[0069] S102. In response to the standardized task request, at least two agents constituting the agent cluster are deployed locally by calling the containerization engine, a directed acyclic graph of tasks is generated based on the standardized task request, and an external storage device is mounted to initialize the project workspace.

[0070] In this embodiment of the application, the global management agent responds to the request by parsing the standardized task request, planning the task process, and generating a directed acyclic graph (DAG) with node dependencies. Each node in the DAG can correspond to an agent with specific capabilities.

[0071] On the directly connected external network-attached storage device, a corresponding project workspace is created and mounted. The containerization engine is invoked locally to deploy multiple project processing containers, and within these containers, various task agents (such as material organization agents, editing agents, and special effects compositing agents) required to form the agent cluster 122 are instantiated. Specifically, based on the acquired DAG graph, the agent cluster 122 schedules the corresponding agent containers sequentially or concurrently with the runtime environment 121 through the containerization engine. After startup, each agent can read the required input data (such as original video footage, model files, etc.) from the directly locally connected network-attached storage device 113 according to its task requirements through the local data synchronization service of the data and state management service 125, and begin executing its dedicated computation or processing work. The transfer of intermediate and final result data between agents can also be carried out through the shared storage or internal network managed by the containerization engine and the runtime environment 121, ensuring the entire process runs efficiently in the local environment.

[0072] S103. Based on the task directed acyclic graph and the preset mapping strategy, each agent in the agent cluster is mapped one-to-one and bound to at least two display terminals connected to the portable computing device, so as to display the subtask execution process of each agent on the respective bound display terminals.

[0073] In this embodiment, the global management agent dynamically binds each agent to a display terminal based on the generated task directed acyclic graph and preset strategies. For example, in a video processing scenario: the material organization agent is bound to the first display terminal to display the material library interface; the editing agent is bound to the second display terminal to display the timeline editing interface; the special effects compositing agent is bound to the third display terminal to display the special effects parameter adjustment interface; simultaneously, the global management agent is bound to the main interactive screen to display the entire process status. After binding, each terminal displays the subtask execution process and status of the corresponding agent in real time.

[0074] In this embodiment, the global management agent continuously detects and collects the status information of each running agent through the interface provided by the containerization engine and the runtime environment 121. This status information includes, but is not limited to: task execution progress percentage, real-time log output stream, key performance indicators (such as CPU / GPU utilization, memory utilization, etc.), and intermediate result previews (such as the current frame in a video clip, a real-time preview image in 3D rendering, etc.). The collected status data is sent to the dynamic mapping module 123 to generate visualization components for each type of data, such as dynamic progress bars, scrollable log windows, real-time updated charts, and image preview windows.

[0075] In this embodiment, the display strategy of the display terminal can be dynamically adjusted according to the operating state of the intelligent agent. For example, when the system detects that the editing intelligent agent is in a computationally intensive phase, it can automatically enlarge the editing intelligent agent on the bound second display terminal screen and highlight its resource consumption chart, etc.

[0076] S104. The agent cluster collaboratively executes subtasks based on the directed acyclic graph of tasks. When the load of the first agent in the agent cluster is detected to be greater than or equal to a preset threshold, at least one subtask undertaken by the first agent is migrated to the second agent in the agent cluster whose load is less than the preset threshold. The display content in the display terminals corresponding to the first agent and the second agent is updated synchronously.

[0077] In this embodiment, the agent cluster 122, under the scheduling of the global management agent, collaboratively executes sub-tasks according to the topological order of the directed acyclic graph of tasks. During this process, the system continuously monitors the load of each agent (e.g., system resource utilization). When the load of the first agent (e.g., the material organization agent) exceeds a preset threshold, migration is triggered: the global management agent migrates some of its sub-tasks to a second agent (e.g., the editing agent) with a lower load. During migration, the display content of the display terminals bound to both agents is synchronously updated; for example, the task to be migrated is mapped from the original display terminal (e.g., the first display terminal) to a new display terminal (e.g., the second display terminal), and the status prompt is updated. For example, for load migration and synchronous update of display content between different agents, refer to... Figure 4 The subtasks of the original first intelligent agent (such as the material organization intelligent agent) (such as the transition effect addition task) are transferred to the second intelligent agent (such as the special effects compositing intelligent agent), and relevant prompt information is displayed on the first display terminal 301 and the third display terminal 303 corresponding to the first and second intelligent agents respectively.

[0078] In this embodiment, the global management agent detects the underlying system resources through the resource detection and subtask migration module 124, receives agent load information reflected on each display terminal, and dynamically allocates the computing resources of the portable computing device 111 to the corresponding agents based on the task load feedback from multiple display terminals. Specifically, the global management agent, through the resource detection and subtask migration module 124, can utilize the CPU cores, GPU computing power, and memory resources allocated to each agent in the containerization engine and runtime environment 121, and can prioritize the computing needs of agents under high load.

[0079] S105. When the process in the directed acyclic graph of a task reaches a preset key decision node, the subsequent process is paused, and a human-computer interaction request is triggered on the display terminal designated as the main interactive screen. Based on the user input of the human-computer interaction, the task parameters are updated, and the execution process in the directed acyclic graph of the task is resumed.

[0080] In this embodiment, when the execution flow of the directed acyclic graph of a task reaches a preset key decision node (e.g., the automatic splicing of the main video segment is completed and awaits confirmation, or a style of special effects to be added needs to be selected), the global control agent pauses the subsequent automated process and triggers a specific human-computer interaction request on the main interactive screen (e.g., displaying preview and confirmation controls, or popping up a style selection list). After receiving user input and updating the task parameters, the system resumes the execution flow of the task graph.

[0081] Specifically, upon detecting a critical node, the global control agent can automatically pause or suspend the execution of subsequent automated processes and trigger at least one of the following methods on a pre-defined global main interactive screen: displaying an interactive interface for parameter configuration or decision confirmation on the main interactive screen; displaying and requesting review of intermediate results or recommended solutions generated by the agent on the main interactive screen. This interactive interface is used to collect user decisions, for example: displaying a rough-cut video and providing buttons such as "Approve / Reject / Mark Time Point"; presenting generated text and providing an editing box and modification suggestion options; or requesting the user to select one from several alternative solutions, etc.

[0082] S106. After the directed acyclic graph task is completed, review and process the output, store the final result in the project workspace, and send a delivery notification.

[0083] In this embodiment, once all nodes in the directed acyclic graph of the task have been executed, the system reviews and processes the final output, stores the final result in the project workspace, and sends a task delivery notification. Specifically, after the user completes the interactive operation on the main interactive screen through the data input device 114, the user's decision data (such as confirmation instructions and modified parameters) is transmitted back to the global management agent. The global management agent can determine the subsequent process based on the user input (such as continuing to execute the next node, jumping to a specific branch, or re-executing the current node), restore the automated execution process of the task objective, until the task objective is completed, and send a delivery notification to the main interactive screen.

[0084] Based on the method shown in steps S101-S106 above, this embodiment of the application displays the intermediate process of automated execution by multiple agents in real time on different display terminals, and shows the task flow of multi-screen visual collaboration in parallel, and introduces user judgment and operation, thereby realizing a collaborative working mode that unifies efficiency and controllability in a portable local computing environment.

[0085] Figure 4 This is a schematic diagram illustrating load migration and synchronized content updates between intelligent agents, as provided in an embodiment of this application. Figure 4 As shown, a global management agent, a material organization agent, an editing agent, and a special effects compositing agent run in the portable computing device 111. The material organization agent corresponds to the first display terminal 301, the editing agent corresponds to the second display terminal 302, the special effects compositing agent corresponds to the third display terminal 303, and the global management agent corresponds to the main interactive screen 304. The first display terminal 301 displays a message such as "Transition effect addition task has been transferred," the second display terminal 302 displays a message such as "Waiting to execute material compositing video clip," the third display terminal 303 displays a message such as "Executing: Transition effect addition task," and the main interactive screen 304 displays a message such as "Current task is being executed: Transition effect addition task."

[0086] like Figure 3 As shown in step S104, the agent cluster (such as the global management agent in the agent cluster) collaboratively executes sub-tasks based on a directed acyclic graph of tasks. When the global management agent detects that the first agent (such as the material organization agent) has too high a load, it migrates some of the sub-tasks of that agent (such as the transition effect addition task) to the second agent (such as the special effects compositing agent) with a lower load. Figure 4 As shown, at the same time as the task migration occurs, the task interface and status information that originally belonged to the material organization agent in the first display terminal 301 are removed simultaneously, and a prompt message is displayed, such as "Transition effect addition task has been transferred". At the same time, a new task execution window is generated on the third display terminal 303 bound to the special effects compositing agent, such as "Executing: Transition effect addition task", thereby realizing the linkage update of the displayed content and the task execution.

[0087] Figure 5 This application provides a flowchart of a multi-screen visual collaboration method tailored to a specific scenario, applicable to, for example... Figure 1 The system framework 100 is shown below. Figure 5 As shown, the method specifically includes the following steps:

[0088] S201. Receive user input and parse user intent.

[0089] In this embodiment, a user's voice input task instruction is received via a voice input device. A locally deployed natural language processing model performs intent recognition on the task instruction and generates a standardized task request based on the intent recognition result. The user inputs the target task, which includes multimedia content processing, via the main interactive screen connected to the portable computing device 111 and its data input device 114 (such as a keyboard or mouse).

[0090] For example, a user inputs via voice: "Create a one-minute product promotional video containing beach scenes from within the last year." The local model of the portable computing device 111 can recognize the command through Voice Activity Detection (VAD) and perform noise reduction and sentence segmentation optimization through Automatic Speech Recognition (ASR). Simultaneously, it performs intent recognition to extract core parameters and verify their integrity. The core parameters include: instructing a full NAS library search in the storage domain (preferably matching the user's own material library by default); filtering data within the current calendar year (e.g., the previous year starting from the current date) in the time parameter; including beach scene tags and geographic location tags in the content; selecting two types of material, such as video clips and photos; determining the processing task, including generating a one-minute highlight montage (balancing the proportion of video and photos), adding timeline-aligned subtitles only to video clips, etc.; and determining the output to be distributed to a multi-screen editing window, etc.

[0091] In this embodiment, the global management agent parses standardized task requests, plans task flows to generate a directed acyclic graph (DAG), and creates a corresponding project workspace for the DAG on an external storage device. Specifically, upon receiving the instruction, the global management agent in the agent cluster 122 uses a built-in domain knowledge model to parse the instruction, decomposing and arranging the macro-intention into an executable DAG. For example, this DAG may sequentially include: the agent for material management collecting and organizing materials, the agent for video rough editing performing preliminary editing, and the agent for special effects compositing performing final compositing and rendering. This DAG defines the dependencies between task flows and agents, as well as preset key human-computer interaction nodes (such as user confirmation after rough editing, receiving user selection of animation style, etc.).

[0092] S202. System initialization and resource deployment.

[0093] In this embodiment, after the global management agent generates the DAG graph, the system initialization process is triggered. Based on the DAG graph requirements, the global management agent prepares isolated runtime environments for each agent using a containerized engine and runtime environment 121. This includes deploying at least one project processing container through the containerized engine, which hosts each agent executing subtasks. The data and state management service 125 can synchronously ensure that the transmission channel between the portable computing device 111 and the directly connected network-attached storage device 113 is ready, preloading or establishing fast access links for materials such as original video files, 3D models, and data tables required by the project. The initial quota of computing resources (such as CPU, GPU, memory, etc.) can be allocated by the global management agent through the resource detection and subtask migration module 124 based on the estimated complexity of the DAG graph. The entire process can be completed in a local offline environment without relying on external cloud computing resources.

[0094] S203. Multi-agent multi-screen parallel collaboration.

[0095] In this embodiment, the global control agent sequentially starts and schedules each agent according to the topological order of the DAG graph. For example, when the tasks of the editing agent and the special effects compositing agent do not depend on each other, they can be scheduled and executed in parallel.

[0096] In this embodiment, the real-time status (progress, logs, resource usage, keyframe previews, etc.) of each activated agent is continuously collected by the dynamic mapping module 123 during execution. Based on a preset binding strategy between agents and display terminals (e.g., binding editing-related agents to a first display terminal, binding graphics and animation-related agents to a second display terminal), the unit globally manages the agents to render and output the designated preview interface of each agent to the designated extended display terminal in real time. Users can thus simultaneously view real-time previews of material organization progress, editing timeline changes, 3D rendering, and data chart generation processes on different display terminals.

[0097] In this embodiment, the global management agent can detect underlying system metrics through the resource detection and subtask migration module 124 and analyze the agent load intensity reflected on each display terminal in real time. Specifically, when the load of the first agent in the agent cluster is detected to be greater than or equal to a preset threshold, the interactive interface and real-time status related to the subtask to be migrated in the first agent are synchronously mapped from the display terminal bound to the first agent to the display terminal bound to the second agent for display. Task migration prompts are displayed on the display terminal bound to the first agent, and the execution status window corresponding to the received subtask is displayed on the display terminal bound to the second agent. For example, when the preview frame rate of the editing agent displayed on the second display terminal decreases and the GPU usage chart is fully loaded, the global management agent dynamically adjusts the resource allocation of the containerization engine and runtime environment 121 through the resource detection and subtask migration module 124, temporarily allocating more GPU computing power from idle agents to the editing agent, thereby achieving dynamic optimization of computing resources.

[0098] For example, the multiple display terminals 112 include four display terminals, namely a first display terminal, a second display terminal, a third display terminal, and a main interactive screen. The material organization agent is bound to the first display terminal, the editing agent is bound to the second display terminal, the special effects compositing agent is bound to the third display terminal, and the global control agent is bound to the main interactive screen.

[0099] In video processing scenarios, the first display terminal shows a media library browsing and filtering interface, the second display terminal shows a video timeline editing interface, and the third display terminal shows a special effects parameter adjustment interface. The main interactive screen comprehensively displays the entire workflow status of the task's directed acyclic graph, the progress of each agent, and the system resource usage. Specifically, this includes:

[0100] The first display terminal is used to display the output of the editing agent, such as the editing software, which imports the mixed video into the timeline, creates a project file, and presets the following tracks: video track (including transition effects), subtitle track (embedded with subtitle files including number sequence and timecode), and photo close-up track (separate layer).

[0101] The second display terminal is used to display the output of the material organization intelligent agent, such as a NAS file manager. For example, the NAS file manager is divided into two columns: the left column displays the original video list (such as with time / tag metadata) and the right column displays the thumbnails of associated photos. It supports dragging and dropping materials across columns to add materials to the editing timeline.

[0102] The third display terminal is used to display the output of the special effects compositing agent, such as a High Dynamic Range (HDR) preview window, allowing real-time viewing of the color and subtitle effects of the montage video. For example, the third display terminal can perform operations such as zooming and subtitle style adjustments (e.g., adjusting font / color / size), and has a built-in color calibration tool to match editing needs.

[0103] The main interactive screen is used to display the output of the global control agent, such as creating project documents, recording the source of materials (e.g., number of videos / photos), processing steps (e.g., model parameters / effect selection), export path, etc., and automatically synchronizing and backing them up to the corresponding folder in the NAS.

[0104] S204. Perform human-computer interaction at critical nodes in the task flow.

[0105] In this embodiment, the global management agent schedules the subtasks corresponding to each node in the directed acyclic graph of tasks in the topological order of the task graph. When the material organization agent, editing agent, or special effects compositing agent completes the execution of the subtask of the upstream node in the task graph, it obtains the corresponding output data and status identifier. Based on the dependency relationship defined by the edges in the task graph, the global management agent uses the output data as input to instruct the material organization agent, editing agent, or special effects compositing agent corresponding to the downstream node to execute the corresponding subtask until the subtask corresponding to each node in the task graph is completed.

[0106] In this embodiment, when the global control agent detects that the execution flow of the task objective has reached a predefined key node in the DAG graph, it triggers human-computer interaction. This triggered human-computer interaction automatically pauses the subsequent automated execution flow and triggers a high-priority modal interaction window on the main interaction screen.

[0107] For example, when the key decision node is the completion of the automatic splicing of the main video segment, the global control agent has been deployed and can provide feedback to the user via Text-to-Speech (TTS). This feedback could include messages such as: eight videos and twelve photos from the beach this year have been selected; a one-minute highlight montage has been generated and subtitles added; the video has been allocated to a multi-screen editing window; and fine-tuning and editing can begin. A preview of the splicing effect and confirmation controls are also displayed on the main interactive screen. When the key decision node is the selection of a video effect style to be added, a list of style candidates and selection controls are displayed on the main interactive screen. Specifically, in the completion of the automatic splicing of the main video segment, a player showing the rough cut video will pop up on the main interactive screen to preview the splicing effect, and confirmation controls such as "Confirm rhythm / Return to modification" and "Mark segments requiring fine-tuning" will be provided to the user. In the selection of a video effect style to be added, multiple stylized rendering candidate lists and selection controls may pop up on the main interactive screen for the user to choose from. The user can use a keyboard, mouse, or graphics tablet to complete judgments and operations on the main interactive screen.

[0108] In this embodiment, the user's decision data is transmitted back in real time. The global control agent resumes the automated execution process based on the decision data and updates the parameters or paths of subsequent tasks based on the user's input (e.g., continue rendering based on the animation style selected by the user, or jump to the retouching stage based on the marked segment).

[0109] S205. Results review and export delivery.

[0110] In this embodiment, after the multi-agent multi-screen parallel collaboration and human-computer interaction nodes are completed, the processing results output by the agent cluster and the corresponding generated review report are displayed on the main interactive screen; and after receiving the user's review confirmation instruction based on the review report, the final result is determined. The system can play the generated final result file (such as the final version of the promotional video) on the main interactive screen for the user to conduct a final review. The user can provide final fine-tuning suggestions at this stage (through the interaction method shown in step S204), and the system can call the corresponding agent to make modifications.

[0111] In this embodiment, after the user confirms the final result, the global management agent can store the finished product file and its corresponding project file to the designated project directory of the locally directly connected network auxiliary storage device 113 through the data and status management service 125. The user can directly share from the NAS or export using a portable device, ensuring that all core data and assets are always in the user's controlled local environment, meeting security and privacy requirements.

[0112] Based on the method described in steps S201-S205 above, this application embodiment constructs a locally deployed multi-screen, multi-agent collaborative system that integrates containerized agent parallel computing, dynamic task scheduling, and multi-screen visual state mapping. While ensuring core data is offline and reducing the risk of leakage, it improves task collaboration efficiency. The system is based on the hardware computing power of portable computing devices. Under the condition of meeting the performance requirements of editing and relying on unified hardware standards and deployment, there is no need to purchase professional collaboration software, thus reducing the overall cost. The portable computing device and the corresponding multiple display terminals can be quickly adapted to various scenarios, and combined with the built-in dynamic fault tolerance mechanism, it improves the efficiency, reliability, and security of human-computer interaction.

[0113] It should be understood that, as mentioned above Figure 5 The steps in the flowcharts are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed; they can be executed in other orders. Furthermore, as mentioned above... Figure 5 The flowchart may include at least some steps or stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0114] Figure 6 This is an interactive timing diagram of a multi-screen visual collaboration method combined with a specific scenario provided in an embodiment of this application. Through the application of... Figure 5 The multi-screen visual collaboration method shown demonstrates the dynamic interaction sequence and parallel relationships among various participants (users, agents, and devices) during the execution of specific tasks in this system. For example... Figure 6 As shown, this sequence involves the user, the global management agent, network-attached storage devices, and multiple agents (including a material organization agent, an editing agent, and a special effects compositing agent, each with its own display terminal) and the main interactive screen. The specific process is as follows:

[0115] 1. Intent parsing phase.

[0116] In this embodiment, a user can submit a video editing task (i.e., input a task request) to the system in natural language (such as speech) via a microphone connected to a portable computing device (as a voice input device). The portable computing device's local speech recognition module first converts the speech into text. The global management agent in the intelligent agent collaboration layer performs deep analysis of the text using a built-in local model. The global management agent extracts key intents and constraints, transforming the recognized task request into structured core parameters, such as: target duration, rhythm requirements, required processing type (including dynamic text and transition effects), associated materials, and external assets (such as specific background music collected by the user). Finally, a machine-readable standardized task request (e.g., JSON format), including parameter validation results, is generated and sent to the global management agent (i.e., parsing the requirements and determining the task flow).

[0117] 2. System initialization and resource mapping phase.

[0118] In this embodiment, upon receiving a standardized task request, the global management agent sends instructions to the containerization engine (e.g., the application container engine Docker) to pull and start the various agent containers required by this embodiment: such as a material organization agent container, an editing agent container (e.g., one encapsulating an editing software engine), and a special effects compositing agent container. The global management agent can also simultaneously start necessary supporting service containers (e.g., file brokers, message buses) and deploy containers such as document collaboration containers and Message Queuing Telemetry Transport (MQTT) containers to ensure service isolation without disrupting the original system environment (i.e., deploying the required containers).

[0119] In this embodiment, the global management agent can execute a preset screen binding strategy through the dynamic mapping module 123 based on the type of agent and task relationships in the DAG graph of the task objective. For example, the detection and operation interface of the material organization agent is bound to the extended display terminal, with the material organization agent corresponding to the first display terminal; the working interface of the editing agent (such as the timeline in editing software) is bound to the extended display terminal, with the editing agent corresponding to the second display terminal; the working interface of the special effects compositing agent is bound to the extended display terminal, with the special effects compositing agent corresponding to the third display terminal; and the detection panel of the global management agent itself is fixedly displayed on the main interactive screen. This mapping information is registered in the system (i.e., the agent is bound to the corresponding display terminal).

[0120] In this embodiment, the global management agent can connect to the mounted network-attached storage device, create a workspace for the project (i.e., mount the media directory), and establish a data channel (i.e., mount the media directory and initialize the project). The system can automatically index or synchronize the user-specified original media and collected background music from the NAS to this space and obtain confirmation information (i.e., confirmation) from the network-attached storage device.

[0121] 3. Multi-agent collaboration and dynamic scheduling stage.

[0122] In this embodiment, the global control agent displays the task flow on the main interactive screen and, based on the DAG graph of the task objective, sequentially or in parallel instructs other agents to execute sub-tasks (including assigning material processing tasks to the material organization agent, editing tasks to the editing agent, and special effects compositing tasks to the special effects compositing agent). Other agents receive and execute the sub-tasks assigned by the global control agent and notify the global control agent upon completion of the tasks.

[0123] For example, the material processing agent sends a retrieval message to the network-attached storage device (i.e., reads / processes materials), displays its operation interface on the first display terminal (i.e. displays the material processing status on the first display terminal), automatically performs intelligent analysis, tagging, grouping, and coarse selection on the original materials, and sends task completion information to the global control agent (i.e., task completed, notifying the next node).

[0124] The editing agent sends a retrieval message to the network-attached storage device (i.e., retrieves the material), opens the project file in the editing software on the second display terminal (i.e., displays the editing status on the second display terminal), performs automatic rough cutting based on the rough selected material, and sends task completion information to the global control agent (i.e., task completion, notifying the next node).

[0125] The special effects compositing agent sends a retrieval message (i.e., retrieves the material) to the network-attached storage device, displays its special effects compositing interface on the third display terminal (i.e., displays the special effects compositing status on the third display terminal), and adds the special effects to be composited to the video material.

[0126] In the embodiments of this application, the work progress bar, real-time log, resource usage dashboard and final results (such as material preview images, timeline screens, etc.) of each intelligent agent can be rendered and displayed on the display terminal it is bound to in real time, realizing the visualization of multi-agent collaboration.

[0127] In this embodiment, the global management agent can continuously monitor the resource load and task queue status of all agents through the resource detection and subtask migration module 124, and adaptively adjust the subtasks executed by the agents. For example, if the global management agent detects that the editing agent is overloaded due to automatic shot analysis (e.g., resource load exceeds a preset load threshold), causing a drop in the preview frame rate on the second display terminal, while the material organization agent has already completed its work and the first display terminal is relatively idle, the global management agent can dynamically decide to migrate the editing agent's subtasks (such as audio waveform analysis or initial shot screening) to an idle material organization agent or a temporarily started general computing agent (i.e., dynamic adjustment; if an agent is overloaded, the subtask is migrated to an idle agent). The global management agent can synchronously update the screen mapping relationship through the dynamic mapping module 123, for example, by adding a window on the first display terminal to display the status of the migrated subtasks, or by providing migration prompts on the second display terminal, ensuring synchronous updates of the visual interface.

[0128] 4. Proactive human-computer interaction stage.

[0129] In this embodiment, when the automated process of the global management agent reaches a preset key node in the DAG graph (i.e., a key node in the task flow, such as the node after the editing agent completes the automatic splicing of all main segments and generates a rough cut), the global management agent can trigger human-computer interaction and automatically pause the subsequent automated effects addition process. The system synchronously triggers a high-priority modal interaction pop-up on the main interactive screen (i.e., automatically triggers the interaction pop-up). For example, the pop-up embeds a player for the rough cut video and provides interactive options to the user. The user can view the display status of the main interactive screen and input decisions / parameters through a data input device (i.e., view and input decisions / parameters), and the global management agent receives feedback user instructions. The global management agent resumes the process and, based on the feedback instructions, determines the subsequent task target DAG graph and continues to execute the method described in the aforementioned multi-agent collaboration and dynamic scheduling stage (i.e., executes the user-instructed operation).

[0130] 5. Review and delivery stage.

[0131] In this embodiment, after the global control agent completes the DAG graph of the task objective, the final result is generated. The system can display the final result on the main interactive screen for user review (i.e., final result review). Users can propose final minor adjustments here (such as modifying the text title), triggering a round of rapid human-computer interaction fine-tuning.

[0132] In this embodiment of the application, after the user confirms the final result, the global management agent can use the data and state management service 125 to completely store the final rendered video file, all project files, logs and version information into the project workspace on the network auxiliary storage device (i.e. store the final result and generate logs), and return a task completion notification to the user (i.e. return a task completion notification).

[0133] Based on the above five stages, such as Figure 6 The interactive sequence diagram shown describes a short video editing scenario, demonstrating how user instructions, parallel computing of intelligent agents, multi-screen visual feedback, dynamic resource scheduling, and human-machine interruption points are coordinated in an orderly manner throughout the entire process from user intent input to local delivery, and how the system achieves multi-screen visual collaboration of multiple intelligent agents. It provides an application method for the method in the embodiments of this application in a specific implementation scenario.

[0134] Figure 7 This is a schematic diagram of a scenario for rapid generation of short video content provided by an embodiment of this application. The task is to create a 30-second promotional short video, and the interface shown by the technical solution of this application on multiple display terminals is illustrated.

[0135] Users can quickly combine product demonstration videos, images, background music, and scripts stored on network-attached storage devices into a short video with basic transitions and subtitles. The specific implementation process is as follows:

[0136] 1. The user issues a voice command instructing the use of video and image materials from a network-attached storage device to generate a 30-second product promotion video based on the text in a document. The portable computing device receives the voice command via a voice input device (such as a microphone). The global control agent in the intelligent agent cluster parses the command using speech recognition and a natural language model, generating a standardized task request containing information such as material paths, target duration, and text content.

[0137] 2. The global control agent responds to the request and executes the following steps in sequence: Creates and mounts a project folder dedicated to this task as the project workspace on the directly connected network auxiliary storage device; invokes the local containerization engine to start three project processing containers, and instantiates the material organization agent, editing agent, and special effects compositing agent respectively; parses the task and generates, for example... Figure 7 The task directed acyclic graph shown on the main interactive screen 304 defines the execution order of the subtasks, which are executed sequentially as follows: material organization and selection (node ​​A), video rough editing and splicing (node ​​B), rough video review (node ​​C), subtitle generation and overlay (node ​​D), and final video review (node ​​E). Dependencies exist between nodes; for example, node C depends on the outputs of nodes A and B.

[0138] 3. Bind multiple agents to the display terminal according to a preset mapping strategy (such as by task type). Specific binding strategies include:

[0139] The media organization agent is responsible for executing the media organization and filtering tasks shown in node A, and is bound to the first display terminal 301. This first display terminal 301 can display a media library browser interface, automatically classifying media and providing keyframe previews;

[0140] The editing agent is responsible for performing the video rough cut and splicing tasks shown at node B, and is bound to the second display terminal 302. The second display terminal 302 displays the video editing timeline and performs shot splicing and music alignment;

[0141] The special effects compositing agent is responsible for executing the task of generating and overlaying subtitles as shown in node D, and is bound to the third display terminal 303. This third display terminal 303 displays the subtitle editor and style template library;

[0142] The global control agent is responsible for executing the rough video review task shown by node C and the final video review task shown by node E. It is bound to the display terminal designated as the main interactive screen 304 to display the overall task progress bar and the resource utilization rate of each agent.

[0143] 4. The intelligent agent cluster is based on a directed acyclic graph (DAG) for controlling and executing tasks by globally managing intelligent agents. For example... Figure 7 As shown, each agent can display prompts on the corresponding display terminal when performing tasks for the node it is responsible for. For example, the material organization agent can display the task performed by node A on the first display terminal 301, that is, display the prompt message "Perform material organization and screening"; the editing agent can display the task performed by node B on the second display terminal 302, that is, display the prompt message "Perform rough cut and splicing of video"; the special effects compositing agent can display the task performed by node D on the third display terminal 303, that is, display the prompt message "Perform generate and overlay subtitles"; and the global control agent can display the tasks performed by nodes C and E on the main interactive screen 304, that is, display the prompt messages "Perform rough cut video review" and "Review final video". Simultaneously display the tasks performed by other intelligent agents on the main interactive screen 304. For example, when the material organization intelligent agent is performing material organization and filtering, display the prompt message "Current task is being performed: material organization and filtering"; when the editing intelligent agent is performing video rough cutting and splicing, display the prompt message "Current task is being performed: video rough cutting and splicing"; when the special effects compositing intelligent agent is performing the task of generating and overlaying subtitles, display the prompt message "Current task is being performed: generating and overlaying subtitles".

[0144] In some implementations, if the GPU load of the editing agent (i.e., the first agent) continuously exceeds a preset threshold (e.g., 80% of the agent's total CPU computing power) during video coarse cutting and splicing, the global management agent can detect this situation and trigger a load balancing strategy to migrate the video coarse cutting and splicing subtask from the editing agent to the effects compositing agent (i.e., the second agent), which has a lower load at the time. During the migration, the parameter panel for video coarse cutting and splicing on the second display terminal 302 disappears, and a new window for video coarse cutting and splicing is simultaneously opened on the third display terminal 303. The entire migration process has no impact on the dependencies of the task graph. For a detailed description of the agent subtask migration, please refer to the foregoing. Figure 4 This will not be elaborated upon here.

[0145] 5. Once the rough cut and splicing of the video are completed automatically, when the workflow reaches a critical decision node (such as node C), the global control agent will pause the subsequent process of generating and overlaying subtitles. An interactive request window will pop up on the main interactive screen 304, for example: the left side will automatically play a preview of the generated rough cut video, and the right side will provide buttons such as "Confirm," "Fine-tuning Segment Order," and "Return to Select Material." After the user views the preview and clicks the "Confirm" button, the system will resume the workflow, and the effects compositing agent will perform the task of generating and overlaying subtitles.

[0146] 6. After all nodes have completed their execution, the global control agent can output the final video file. The system saves it to the project workspace on the network-attached storage device and sends a notification message to the user, such as "Video compositing is complete, and the file has been saved to the project workspace."

[0147] Example 3:

[0148] Figure 8 This is a schematic diagram of the module structure of a multi-screen visual collaboration system based on intelligent agents, provided in an embodiment of this application, used to implement the multi-screen visual collaboration method based on intelligent agents as shown in this application. Figure 8 As shown, the agent-based multi-screen visual collaboration system 800 specifically includes:

[0149] The portable computing device 111 integrates multiple display interfaces, serving as the core computing and scheduling unit. The portable computing device 111 is equipped with functional modules for executing the steps of the aforementioned agent-based multi-screen visual collaboration method.

[0150] Multiple display terminals 112 are connected to multiple display interfaces on the portable computing device 111, and a main interactive screen is designated in the multiple display terminals 112 for visually presenting the display interface and system status of different intelligent agents.

[0151] A network-attached storage device 113 is communicatively connected to a portable computing device 111 to provide project workspace and material storage.

[0152] It is understood that the functional division between the modules illustrated in the embodiments of this application is merely illustrative and does not constitute a limitation on the functionality of the agent-based multi-screen visual collaboration system 800. In other embodiments of this application, the agent-based multi-screen visual collaboration system 800 may also employ different modules or combinations of multiple modules to implement the functionality of the agent-based multi-screen visual collaboration system 800.

[0153] Figure 9 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. The computer device 900 may include the aforementioned... Figure 1 The system framework 100 shown and the aforementioned Figure 8 The illustrated example is an agent-based multi-screen visual collaboration system 800. Figure 9 As shown, the computer device 900 includes: a processor 901, a memory 902, a communication module 904, and a computer program 903 stored in the memory 902 and executable on the processor 901. When the processor 901 executes the computer program 903, it implements the aforementioned... Figure 3 , Figure 5 and Figure 6 The execution steps are shown. For example, the computer program 903 described above can be divided into one or more units / modules, which are stored in the memory 902 and executed by the processor 901 to complete this application.

[0154] The aforementioned one or more units / modules may be a series of computer program instruction segments capable of performing a specific function. These instruction segments describe the execution process of the aforementioned computer program 903 within the aforementioned computer device 900. For example, the aforementioned computer program 903 may be used to perform actions such as... Figure 3 Steps S101-S106, and such as Figure 5 The specific functions or mechanisms of the methods shown in steps S201-S205 have been described in the above embodiments and will not be repeated here.

[0155] Those skilled in the art will understand that Figure 9 This is merely an example of computer device 900 and does not constitute a limitation on computer device 900. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device 900 described above may also include input / output devices, network access devices, buses, etc.

[0156] The processor 901 mentioned above can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0157] In some embodiments, the processor 901 may include one or more interfaces. These interfaces may include: an internal integrated circuit I2C interface, an integrated circuit built-in audio bus I2S interface, a pulse code modulation (PCM) interface, a universal asynchronous transceiver (URAT) interface, a mobile industry processor (MIPI) interface, a general purpose input / output (GPIO) interface, and / or a universal serial bus (USB) interface, etc. It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the computer device 900. In other embodiments of this application, the computer device 900 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0158] In some embodiments, the computer device 900 can connect internal devices and modules via one or more interfaces. The aforementioned memory 902 can be an internal storage unit of the computer device 900, such as a hard disk or RAM. The aforementioned memory 902 can also include both internal storage units and external storage devices. The aforementioned memory 902 is used to store the computer program and other programs and data required by the computer device 900. The aforementioned memory 902 can also be used to temporarily store data that has been output or will be output.

[0159] The communication module 904 can provide solutions for wireless communication applications on the computer device 900, including wireless local area networks (WLAN), Bluetooth (BT), Global Navigation Satellite System (GNSS), FM, Near Field Communication (NFC), and Infrared (IR). The communication module 904 can be one or more devices integrating at least one communication processing module. The communication module receives electromagnetic waves via an antenna, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 901. The communication module 904 can also receive signals to be transmitted from the processor 901, frequency-modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0160] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above equipment can be divided into different functional units or modules to complete all or part of the functions described above.

[0161] The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of software functional units.

[0162] In the embodiments of this application, the specific names of each functional unit and module are only for easy distinction and are not intended to limit the scope of protection of this application. It should be understood that each step in the above-described method embodiments provided in this application can be completed by the integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0163] This application also provides a computer program product, which includes: a computer program (also referred to as code or instructions) that, when run, causes a computer to execute the agent-based multi-screen visual collaboration method described in the above embodiments.

[0164] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0165] In the embodiments provided in this application, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, in the form of a computer program product.

[0166] The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0167] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the computer device in any of the foregoing embodiments.

[0168] Figure 10This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. Figure 10 As shown, the computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0169] The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Universal Optical Discs, DVDs), or semiconductor media (e.g., solid-state drives, SSDs), etc.

[0170] Those skilled in the art will understand that implementing all or part of the processes in the foregoing embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the foregoing method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as readable storage devices or random access memory, magnetic disks, or optical disks.

[0171] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A multi-screen visual collaboration method based on intelligent agents, characterized in that, Applied to portable computing devices, the method includes: Receive and parse user-input task instructions to generate standardized task requests; In response to the standardized task request, at least two agents constituting the agent cluster are deployed locally by calling the containerization engine, a directed acyclic graph of tasks is generated based on the standardized task request, and an external storage device is mounted to initialize the project workspace. Based on the task directed acyclic graph and the preset mapping strategy, each agent in the agent cluster is mapped one-to-one and bound to at least two display terminals connected to the portable computing device, so that the subtask execution process corresponding to each agent is displayed on the respective bound display terminals. The intelligent agent cluster collaboratively executes the sub-tasks based on the directed acyclic graph of the task. When the load of the first intelligent agent in the intelligent agent cluster is detected to be greater than or equal to a preset threshold, at least one of the sub-tasks undertaken by the first intelligent agent is migrated to the second intelligent agent in the intelligent agent cluster whose load is less than the preset threshold. The display content in the display terminals corresponding to the first intelligent agent and the second intelligent agent is updated synchronously. When the process in the directed acyclic graph of the task reaches a preset key decision node, the subsequent process is paused, and a human-computer interaction request is triggered in the display terminal designated as the main interactive screen. Based on the user input of the human-computer interaction, the task parameters are updated, and the execution process in the directed acyclic graph of the task is restored. After the directed acyclic graph task is completed, the output is reviewed and processed, and the final result is stored in the project workspace and a delivery notification is sent.

2. The method according to claim 1, characterized in that, The process of receiving and parsing user input task instructions specifically includes: The task instructions are received by the user's voice input via a voice input device; The task instructions are subjected to intent recognition by a locally deployed natural language processing model, and the standardized task request is generated based on the intent recognition results.

3. The method according to claim 2, characterized in that, The intelligent agent cluster includes a global management intelligent agent. The process of generating a directed acyclic graph of tasks based on the standardized task requests and mounting external storage devices to initialize the project workspace is specifically executed by the global management intelligent agent, including: The global management agent parses the standardized task request, plans the task flow to generate the task directed acyclic graph, and creates the corresponding project workspace for the task directed acyclic graph on the external storage device.

4. The method according to claim 3, characterized in that, The deployment of at least two agents constituting the agent cluster by locally calling the containerization engine specifically includes: The global management agent deploys at least one project processing container through a containerization engine, and the project processing container is used to host each agent that executes the sub-task.

5. The method according to claim 4, characterized in that, The intelligent agent cluster also includes a material organization intelligent agent, an editing intelligent agent, and a special effects compositing intelligent agent. The step of mapping and binding each intelligent agent in the intelligent agent cluster to at least two display terminals connected to the portable computing device specifically includes: The material organization agent is bound to the first display terminal, the editing agent is bound to the second display terminal, the special effects compositing agent is bound to the third display terminal, and the global management agent is bound to the main interactive screen.

6. The method according to claim 5, characterized in that, The step of displaying the subtask execution process corresponding to each intelligent agent on the respective bound display terminal specifically includes: In a video processing scenario, the first display terminal displays a material library browsing and filtering interface, the second display terminal displays a video timeline editing interface, the third display terminal displays a special effects parameter adjustment interface, and the main interactive screen comprehensively displays the entire process status of the task's directed acyclic graph, the progress of each agent, and the system resource usage.

7. The method according to claim 6, characterized in that, The intelligent agent cluster collaboratively executes the sub-tasks based on the directed acyclic graph of the task, specifically including: The global management agent schedules the subtasks corresponding to each node in the directed acyclic graph of tasks in sequence according to the topological order of the directed acyclic graph of tasks. After the material organization agent, the editing agent, or the special effects compositing agent completes the subtask of the upstream node in the directed acyclic graph of the task, the corresponding output data and status identifier are obtained. The global management agent, based on the dependency relationships defined by the edges in the directed acyclic graph of the task, takes the output data as input and instructs the material organization agent, the editing agent, or the special effects compositing agent corresponding to the downstream node to execute the corresponding sub-task, until the sub-task corresponding to each node in the directed acyclic graph of the task is completed.

8. The method according to claim 7, characterized in that, The first intelligent agent and the second intelligent agent are any two different intelligent agents among the global management intelligent agent, the material organization intelligent agent, the editing intelligent agent, and the special effects compositing intelligent agent. Updating the display content in the display terminal correspondingly bound to the first intelligent agent and the second intelligent agent specifically includes: The interactive interface and real-time status related to the subtask to be migrated in the first intelligent agent are synchronously mapped from the display terminal bound to the first intelligent agent to the display terminal bound to the second intelligent agent for display. The task migration prompt information is displayed on the display terminal bound to the first intelligent agent, and the execution status window corresponding to receiving the sub-task is displayed on the display terminal bound to the second intelligent agent.

9. The method according to claim 1 or 8, characterized in that, The key decision nodes include the automatic completion node for video main segment splicing and the selection node for video effects style to be added. Triggering a human-computer interaction request in the display terminal designated as the main interactive screen specifically includes: When the key decision node is the node where the main video segment is automatically spliced, a splicing effect preview and confirmation control are displayed on the main interactive screen. When the key decision node is the video effect style selection node to be added, a style candidate list and selection control are displayed on the main interactive screen.

10. A multi-screen visual collaboration system based on intelligent agents, characterized in that, The system for implementing the method as described in any one of claims 1-9 comprises: Portable computing devices serve as the core computing and scheduling units; An external storage device, communicatively connected to the portable computing device, is used to provide project workspace and material storage; At least two display terminals are connected to the portable computing device for visually presenting the display interfaces and system status of different intelligent agents; The portable computing device stores computer instructions, which, when executed by the portable computing device, implement the method steps as described in any one of claims 1-9.

Images